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Preface 


This  volume  contains  the  proceedings  of  the  Fourth  Workshop  on  Hybrid  Sy¬ 
stems:  Computation  and  Control  (HSCC  2001)  held  in  Rome,  Italy  on  March 
28-30,  2001.  The  Workshop  on  Hybrid  Systems  attracts  researchers  from  indu¬ 
stry  and  academia  interested  in  modeling,  analysis,  synthesis,  and  implementa¬ 
tion  of  dynamic  and  reactive  systems  involving  both  discrete  (integer,  logical, 
symbolic)  and  continuous  behaviors.  It  is  a  forum  for  the  discussion  of  the  la¬ 
test  developments  in  all  aspects  of  hybrid  systems,  including  formal  models  and 
computational  representations,  algorithms  and  heuristics,  computational  tools, 
and  new  challenging  applications. 

The  Fourth  HSCC  International  Workshop  continues  the  series  of  workshops 
held  in  Grenoble,  France  (HART’97),  Berkeley,  California,  USA  (HSCC’98),  Nij¬ 
megen,  The  Netherlands  (HSCC’99),  and  Pittsburgh,  Pennsylvania,  USA  (HSCC 
2000) .  Proceedings  of  these  workshops  have  been  published  in  the  Lecture  Notes 
in  Computer  Science  (LNCS)  series  by  Springer- Verlag. 

In  line  with  the  beautiful  work  that  led  to  the  design  of  the  palace  in  which 
the  workshop  was  held,  Palazzo  Lancellotti  in  Rome,  resulting  from  the  colla¬ 
boration  of  many  artists  and  architects  of  different  backgrounds,  the  challenge 
faced  by  the  hybrid  system  community  is  to  harmonize  and  extract  the  best  from 
two  main  research  areas:  computer  science  and  control  theory.  Terminology,  ma¬ 
thematical  tools,  and  abstractions  are  different,  problems  considered  relevant  by 
one  community  may  be  considered  trivial  by  the  other,  yet  it  is  this  very  diffe¬ 
rence  that  may  bring  new  vistas  to  traditional  research  fields  to  escape  the  trap 
of  routine.  The  steering  committee  of  the  workshop  series  has  been  appointed  to 
guide  the  directions  of  the  research  in  troubled  water  balancing  the  membership 
among  computer  scientists,  control  theorists,  and  application  experts.  The  tech¬ 
nical  program  committee  has  been  assembled  following  the  same  principle.  The 
committee  has  done  a  wonderful  job  in  reviewing  and  discussing  82  submissions 
(a  record  number  since  the  inception  of  the  workshop  series).  All  requested  re¬ 
views  were  received  (a  world- wide  record  among  all  workshops!).  After  extended 
and,  at  times,  intense  discussions,  36  papers  were  selected  for  presentation  at  the 
workshop  and  publication  in  this  volume.  While  the  technical  quality  of  the  pa¬ 
pers  is  excellent,  we  cannot  underestimate  the  preponderance  of  control  theory 
papers  and  the  scarcity  of  application  papers.  The  theory  papers  are  mainly  di¬ 
rected  at  the  consolidation  of  the  foundations  of  the  field,  a  hardly  unexpected 
outcome  in  an  area  that  is  approaching  a  new  level  of  maturity.  However,  the 
lack  of  relevant  application  papers  is  somewhat  worrisome.  For  this  reason,  we 
preferred  to  give  emphasis  to  applications  in  the  invited  papers  to  the  workshop: 
Manfred  Morari  (ETH  Zurich),  Costas  Pantelides  (Imperial  College),  and  Janos 
Sztipanovits  (Vanderbilt  University)  are  all  well  known  for  their  work  in  hybrid 
system  applications  and  in  embedded-system  design.  In  addition,  we  included 
in  the  workshop  a  panel  on  applications  of  hybrid  systems.  The  participants  to 


VI 


Preface 


the  panel  addressed  the  challenges  of  using  a  richly  expressive  theory,  being,  as 
such,  relatively  poor  in  computationally  affordable  synthesis  and  analysis  tools, 
to  yield  relevant  results  in  the  real-life  world.  They  also  addressed  the  issue  of 
merging  knowledge  about  tools  and  methods  in  control  and  computer  science 
so  that  we  may  avoid  the  risk  of  re-inventing  in  one  field  results  that  are  well 
known  in  the  other. 

We  believe  that  embedded  systems  will  be  the  main  application  vehicle  for 
our  technology  and  as  such  deserve  particular  attention.  Embedded  systems  will 
also  be  the  main  application  domain  for  electronics  in  general.  Since  embedded 
systems  require  design  methods  that  guarantee  correct  and  efficient  behavior  in 
harsh  environments,  a  strong  theoretical  approach  to  synthesis  and  verification 
is  badly  needed.  They  are  hybrid  in  nature:  continuous  and  discrete  mix  freely 
in  a  variety  of  application  domains.  Software  and  control  will  play  a  dominant 
role.  Hence,  we  believe  that  our  community  will  be  an  important  constituency 
in  founding  the  field  of  embedded  system  theory  and  design. 

We  wish  to  thank  the  organizations  (PARADES,  Progetto  Finalizzato  Ma- 
dess  II,  Consiglio  Nazionale  delle  Ricerche,  Army  Research  Office,  National 
Science  Foundation)  that  financially  supported  the  workshop.  Moreover,  we  ack¬ 
nowledge  the  contribution  of  Magneti-Marelli,  an  automotive  electronics  com¬ 
pany  that  has  put  to  good  use  hybrid  system  technology  in  its  products.  In 
particular,  the  support  and  continuous  encouragement  of  Dr.  Daniele  Pecchini, 
President  and  General  Manager  of  Magneti  Marelli  Powertrain  Division,  is  ack¬ 
nowledged.  We  thank  Prof.  Richard  Gerber  for  letting  us  use  START,  his  soft¬ 
ware  conference  manager. 

The  final  remark  is  dedicated  to  the  Organizing  Committee,  whose  members 
spent  long  hours  making  sure  everything  was  correctly  handled,  from  call  for  pa¬ 
pers  to  hotel  information,  and  paper  submission.  In  particular,  Andrea  Balluchi 
and  Luca  Benvenuti  have  spent  an  inordinate  amount  of  time  coping  with  the 
software,  trying  to  keep  all  the  web  material  in  synch  and  making  sure  authors 
submitted  the  correct  versions  of  their  papers  and  the  appropriate  documents 
that  E-conomy  bureaucracy  imposes  on  us. 
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Abstract.  We  envision  the  role  of  control  to  expand  rapidly  in  two  di¬ 
rections.  It  will  impact  novel  application  areas,  which  have  yet  to  benefit 
from  the  power  of  feedback,  and,  as  an  embedded  technology,  control 
will  extend  its  reach  far  beyond  the  traditional  narrow  concept  to  in¬ 
clude  higher  level  functions  of  operation.  Our  research  program  is  built 
on  this  vision.  Eventually,  these  ideas  should  also  radically  change  what 
is  taught  in  our  class  rooms,  so  that  our  students  can  transfer  these 
techniques  to  industry  effectively  and  reap  its  benefits. 

In  all  control  applications  the  actual  control  algorithm  is  just  one  tiny 
part  of  the  overall  system  designed  to  ensure  safe,  reliable  and  economi¬ 
cal  operation.  Success  or  failure  of  "operation”  are  attributable  at  least 
as  much  to  "the  rest"  as  to  the  control  algorithm  itself.  At  the  lowest 
level  the  control  algorithm  is  endowed  with  functionality  to  deal  with  op¬ 
erating  constraints  and  to  switch  smoothly  between  different  operating 
regimes.  At  the  highest  levels  the  control  algorithm  may  be  embedded 
in  a  scheduling  system  or  even  an  Enterprise  Resource  Planning  (ERP) 
system.  At  all  levels  this  embedding  creates  a  heterogeneous  system  com¬ 
prised  of  many  interacting  subsystems,  typically  referred  to  as  a  hybrid 
system. 

The  integration  should  eventually  lead  to  a  safer,  smoother,  more  re¬ 
sponsive  and  more  competitive  functioning  of  the  entire  system  or  or¬ 
ganization.  About  three  years  ago  we  embarked  on  a  major  research 
program  toward  this  goal.  Its  objective  is  the  development  of  new  theo¬ 
retical  tools  to  model,  analyze,  simulate  and  control  such  large  complex 
hybrid  systems  involving  continuous  and  discrete  states,  whose  behavior 
is  governed  by  dynamics,  logical  statements  and  constraints.  In  this  talk 
we  will  summarize  the  highlights  and  try  to  put  them  in  perspective. 
Modeling  and  Simulation:  The  models  should  facilitate  the  analysis  and, 
at  the  same  time,  capture  the  complex  behavior,  that  hybrid  systems  are 
known  to  exhibit.  Based  on  these  considerations,  we  introduced  a  discrete 
time  description,  combining  linear  dynamics  with  Boolean  variables.  This 
mixed  logical  dynamical  form  (MLD)  form  is  capable  to  model  a  broad 
class  of  systems  arising  in  many  applications  from  the  automotive,  air¬ 
craft,  chemical  and  information  technology  fields.  Supply  chains  used  in 
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business  models  can  be  conveniently  modeled  as  MLD  systems  as  well. 
We  defined  a  new  modelling  language  (HYSDEL)  and  wrote  a  compiler 
to  assist  the  user  in  the  formulation  of  MLD  models. 

Controller  Synthesis:  For  controller  synthesis  we  formulate  a  finite  hori¬ 
zon  optimal  control  problem  and  apply  the  result  in  a  moving  horizon 
fashion.  For  MLD  models  the  optimization  problem  is  a  mixed-integer 
linear  program  (MILP)  which  must  be  solved  in  real  time  at  each  sam¬ 
pling  time.  We  have  proven  that  the  resulting  state  feedback  control  law 
is  piece- wise  linear  over  a  polyhedral  partition  of  the  state  space.  As  an 
alternative  to  on-line  optimization,  we  can  determine  this  control  law 
explicitly  by  solving  a  multi-parametric  MILP. 

State  Estimation  and  Fault  Detection:  For  application  of  the  described 
control  law  the  system  states  must  be  known.  Estimation  of  the  states 
of  an  MLD  system  is  a  complex  nonlinear  filtering  problem.  We  have 
defined  a  moving  horizon  estimator,  where  at  each  time  step  a  mixed  in¬ 
teger  quadratic  program  must  be  solved  to  arrive  at  the  state  estimates. 
We  have  proven  the  convergence  of  the  estimator  if  certain  observability 
properties  are  satisfied.  Complex  fault  situations  can  be  modeled  accu¬ 
rately  in  the  MLD  framework.  Fault  detection  is  another  application  of 
the  new  estimator. 
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Abstract.  Most  processes  of  practical  interest  are  hybrid  in  nature,  ex¬ 
hibiting  both  continuous  and  discrete  characteristics.  In  many  cases,  the 
hybrid  behaviour  is  a  result  of  intrinsic  physical  phenomena  that  lead 
to  (practically)  instantaneous  events  such  as  the  appearance  and  disap¬ 
pearance  of  thermodynamic  phases,  changes  in  flow  regimes,  equipment 
failures  etc.  All  such  events  effect  qualitative  changes  in  the  underlying 
continuous  dynamics,  thereby  leading  to  hybrid  macroscopic  behaviour. 
In  other  cases,  the  hybrid  nature  arises  from  external  discrete  actions 
imposed  on  the  process  by  its  control  system.  For  example,  the  latter 
may  apply  quantisation  to  convert  continuous  process  measurements  into 
discrete  ones  and/or  continuous  control  outputs  into  discrete  actions. 
Hybrid  processes  and  hybrid  controllers,  and  their  combination,  can  be 
modelled  in  terms  of  State- Transition  Networks  (STNs).  The  system  be¬ 
haviour  in  each  state  is  described  by  a  different  set  of  continuous  equa¬ 
tions  (typically  a  mixed  system  of  partial  and/or  ordinary  differential 
and  algebraic  equations).  At  any  particular  time  during  its  operation, 
the  system  is  in  exactly  one  such  state.  An  instantaneous  transition  to  a 
different  state  may  take  place  if  a  certain  logical  condition  becomes  true. 
Each  transition  is  also  characterised  by  a  set  of  continuous  relations  that 
determine  unique  values  for  the  system  variables  immediately  following 
the  transition  in  terms  of  their  values  immediately  preceding  it. 

In  this  presentation,  we  consider  mathematical  formulations  and  tech¬ 
niques  for  the  optimisation  of  hybrid  systems  described  by  STNs.  This 
generally  seeks  to  determine  the  time  variation  of  a  set  of  controls  and/or 
the  values  of  a  set  of  time-invariant  parameters  that  optimise  some  as¬ 
pect  of  the  dynamic  behaviour  of  the  system.  The  time  horizon  of  interest 
may  be  fixed  or  variable,  subject  to  specified  lower  and  upper  bounds. 
The  equations  that  determine  the  system  behaviour  in  each  state  may 
be  augmented  with  additional  inequality  constraints  imposing  certain 
restrictions  (related  to  safety  or  operability)  on  the  acceptable  system 
trajectories.  The  objective  function  to  be  minimised  or  maximised  is  usu¬ 
ally  a  combination  of  fixed  contributions  (depending  on  the  values  of  the 
time-invariant  parameters)  and  variable  contributions  (depending  on  the 
system  trajectory,  including  the  variation  of  the  controls). 

As  an  illustration,  we  start  with  simple  linear  systems  operating  in  the 
discrete  time  domain,  possibly  involving  uncertain  parameters.  We  then 
proceed  to  consider  the  more  complex  problem  of  the  optimisation  of 
nonlinear  hybrid  systems  operating  in  the  continuous  time  domain. 
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Abstract.  One  of  the  most  pervasive  applications  of  computing  is  infor¬ 
mation  processing  tightly  integrated  with  physical  processes.  Embedded 
computing  rapidly  takes  over  the  role  of  being  a  universal  integrator  for 
physical  systems.  This  trend  is  based  on  a  fundamental  technical  reason: 
digital  information  processing  is  uniquely  suitable  for  controlling  and 
implementing  complex  interactions  among  physical  system  components. 
The  expanding  integration  role  of  computing  challenges  the  state-of-the- 
art  in  both  system  and  software  design.  First,  the  traditional  separation 
of  related  design  disciplines  is  not  maintainable.  Predictability  of  the 
design  requires  integrated  modeling  and  analysis  of  physical  processes 
and  information  processing.  Second,  the  narrow  focus  of  current  software 
technology  on  functional  composition  is  not  sufficient.  Essential  physi¬ 
cal  properties  of  embedded  computing  systems,  such  as  timing,  noise  or 
fault  behavior,  cut  across  functional  boundaries,  which  makes  software 
design  and  implementation  extremely  hard  and  expensive.  Third,  design 
technologies,  which  are  based  on  the  modeling  and  analysis  of  systems 
with  static  structure,  are  becoming  inadequate.  Although  networked  em¬ 
bedded  computing  combined  with  inexpensive  MEMS-based  sensors  and 
actuators  make  the  construction  of  large  physical  systems  with  continu¬ 
ously  changing  structure  and  physical  interactions  feasible,  their  design 
is  an  open  challenge. 

The  first  part  of  the  talk  provides  an  overview  of  the  unique  challenges 
and  new  research  directions  in  embedded  system  and  software  design. 
The  second  part  of  the  talk  describes  the  Model-Integrated  Computing 
(MIC)  approach  to  address  some  of  these  challenges.  Using  the  design 
of  structurally  adaptive  embedded  processing  systems  as  example,  the 
following  three  topics  will  be  covered: 

1 .  Methods  and  tools  for  the  specification  and  construction  of  multiple- 
view,  domain-specific  modeling  languages  and  integrated  design  en¬ 
vironments.  The  MIC  approach  is  based  on  the  application  of  meta¬ 
modeling,  meta-programmable  modeling  tools  and  model  translators 
that  form  the  foundation  for  composable  design  environments. 

2.  Automated  synthesis  of  processing  architectures  satisfying  multiple 
functional  and  physical  constraints.  The  method  described  is  based 
on  symbolic  constraint  satisfaction. 

3.  Application  of  generative  programming  techniques  with  special  em¬ 
phasis  on  model-based  software  generators. 
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Hybrid  systems  are  richly  expressive  models  for  a  large  variety  of  potential  ap¬ 
plications.  However,  being  so  rich  as  to  include  continuous  nonlinear  dynamical 
systems,  discrete-event  systems  and  other  models  of  computation  (finite-state 
machines  and  data  fiow  come  to  mind  here),  they  are  not  amenable  to  com¬ 
putationally  attractive  techniques  for  synthesis  and  analysis  and  present  hard 
numerical  problems  to  simulation.  Hence,  applying  the  methods  typical  of  this 
technology  requires  non  trivial  amount  of  approximation  and  abstraction.  And 
approximation  and  abstraction  are  effective  only  if  the  domain  of  application 
is  deeply  understood.  Thus,  significant  applications  of  hybrid  systems  require  a 
great  deal  of  work  both  to  select  the  right  abstraction  level  and  to  derive  algo¬ 
rithms  that  exploit  the  particularities  of  the  domain  of  application.  In  addition, 
one  needs  to  motivate  and  document  convincingly  why  using  hybrid  systems  can 
yield  better  results  than  other  techniques.  In  this  respect,  there  has  been  an  on¬ 
going  debate  as  to  what  constitutes  a  meaningful  result  in  applications:  on  one 
hand,  novel  languages  for  describing  hybrid  systems  and  capturing  their  prop¬ 
erties  may  be  considered  sophomoric  exercises  by  experts  in  languages,  on  the 
other,  formal  verification  tools  that  in  general  can  handle  small  systems  may  be 
seen  as  toys  for  who  is  trying  to  tame  entire  chemical  plants.  On  the  simulation 
front,  how  to  deal  with  discontinuities  of  trajectories  is  a  major  issue.  Numerical 
analysts  have  been  looking  at  these  problems  only  recently  and  with  a  great  deal 
of  skepticism  as  to  what  can  be  proven  rigorously.  Hybrid  system  researchers  are 
now  getting  seriously  in  the  simulation  arena  exploiting  what  has  been  done  in 
the  numerical  analysis  arena. 

The  goal  of  the  panel  is  to  bring  experts  from  the  two  reference  communi¬ 
ties  of  hybrid  systems  (computer  science  and  control)  to  debate  whether  hybrid 
system  applications  can  indeed  be  compelling  and  what  can  be  done  to  prevent 
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naive  work  on  both  sides  when  straddling  across  competence  domains.  Simula¬ 
tion  and  verification  in  general  will  also  be  discussed  in  the  frame  of  the  work 
done  in  numerical  analysis.  Predicting  the  outcome  of  the  panel,  we  would  like 
to  end  with  a  positive  note:  hybrid  system  technology  is  relevant  to  important 
applications  but  it  has  to  be  handled  with  great  care  and  pushing  the  cart  all  in 
the  same  directions  will  give  the  hybrid  system  community  the  relevance  it  has 
the  right  of  aspiring  to. 


Design  of  Luenberger  Observers  for  a  Class  of 
Hybrid  Linear  Systems 


A.  Alessandri^  and  P.  Coletta^’^ 

^  Naval  Automation  Institute,  lAN-CNR  National  Research  Council  of  Italy, 
Via  De  Marini  6,  16149  Genova,  Italy 
angeloQian . ge . cnr . it 

^  Department  of  Communications,  Computer  and  System  Sciences, 
DIST-University  of  Genoa,  Via  Opera  Pia  13,  16145  Genova,  Italy 
paoloQian . ge . cnr . it 


Abstract.  An  approach  to  estimation  for  a  class  of  hybrid  discrete-time 
linear  systems  using  Luenberger  observers  is  presented.  The  proposed 
Luenberger  observer  for  such  a  kind  of  systems  relies  on  the  switching 
among  different  gains.  Convergence  conditions  have  been  found  to  ensure 
the  stability  of  the  error  dynamics  and  the  related  gains  may  be  selected 
by  solving  a  set  of  linear  matrix  inequalities  (LMIs).  Moreover,  this  ob¬ 
server  may  be  improved  by  suitably  updating  the  estimate  using  the  last 
measures.  This  update  enables  one  to  reduce  the  norm  of  the  estimation 
error  and  is  based  on  the  so-called  projection  method.  Simulation  results 
are  reported  to  show  the  effectiveness  of  these  methods  in  the  estimation 
for  hybrid  discrete-time  linear  systems. 


1  Introduction 

Hybrid  systems  have  recently  gained  a  great  attention  and  the  research  in  this 
area  has  been  devoted  more  to  control  problems.  In  this  work,  the  subject  is  the 
state  estimation  for  a  class  of  hybrid  systems  described  by  switching  discrete-time 
linear  equations.  Switching  systems  are  well-suited  to  dealing  with  applications 
like,  for  example,  gain  scheduling,  reconfigurable  control,  and  fault  diagnosis, 
which  enable  one  to  point  out  the  importance  of  constructing  observers  for  such 
systems. 

The  problem  of  estimating  the  state  of  a  switching  system  was  originally 
stated  in  [1] .  Later  on,  a  lot  of  researches  investigated  the  issues  related  to  such 
a  problem  in  a  probabilistic  framework,  i.e.,  supposing  that  the  transitions  occur 
according  to  a  model  described  by  a  first-order  Markov  chain.  Difficulties  may 
arise  in  the  solution  of  optimal  Bayesian  estimation  problems  and  the  interested 
reader  is  referred,  among  others,  to  [2]  and  [3].  Another  relevant  topic  concerns 
the  so-called  multi-model  estimation.  Such  a  subject  is  quite  vast  and  involves 
many  application-oriented  problems  (for  an  introduction,  see  [4]).  Summing-up, 
all  the  above-mentioned  approaches  rely  on  a  stochastic  setting  and  the  switching 
event  is  supposed  unknown.  Here,  we  focus  on  the  problem  of  estimating  the 
state  of  a  switching  system  by  assuming  to  know  both  time  and  mode  of  the 
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switching.  Nevertheless,  also  in  this  context,  the  problem  remains  hard  to  solve 
for  the  difficulties  of  both  guaranteeing  the  stability  of  the  estimation  error  and 
devising  a  suitable,  efficient  observer  design  procedure.  It  is  worth  noting  that 
we  will  make  no  assumption  on  the  probabilistic  description  of  the  system  mode 
transitions  to  derive  the  stability  results  of  the  proposed  estimation  methods. 

Gain  switching  observers  for  continuous-time  nonlinear  systems  have  been 
considered  in  [5],  where  stable  switching  laws  are  searched  for  with  different 
Lyapunov  functions  for  each  gain.  A  different  approach  based  on  coprime  factor¬ 
ization  is  proposed  in  [6]  to  construct  an  observer  for  switching  continuous-time 
linear  systems.  In  the  present  paper,  the  goal  is  to  find  an  estimator  with  a 
stable  estimation  error  in  the  presence  of  any  switching  in  a  given  finite  set  of 
admissible  system  modes.  Such  a  problem  turns  out  to  be  more  difficult  than 
the  standard  design  of  Luenberger  observers  for  time-invariant  linear  systems. 
In  this  case,  a  Luenberger  observer  provides  a  convergent  error  dynamics  if  and 
only  if  the  gain  is  chosen  such  that  the  poles  of  the  error  dynamics  are  in  the 
strictly  stable  region.  This  condition  is  not  sufficient  to  ensure  the  stability  of 
the  estimation  error  for  a  switching  linear  system.  The  gain  selection  of  a  switch¬ 
ing  observer  is  nontrivial  as  it  involves  the  typical  stability  issues  of  the  hybrid 
systems  (for  an  introduction,  see  [7],  [8],  and  [9]).  In  our  case,  the  solution  of 
this  problem  has  been  addressed  by  seeking  a  common  Lyapunov  function.  This, 
in  turn,  can  be  reduced  to  the  fulfillment  of  linear  matrix  inequalities  (LMIs), 
which  allow  one  to  easily  obtain  a  solution  in  a  computationally  feasible  way. 

An  improvement  to  this  Luenberger  observer  has  been  made  by  applying  a 
projection  method  [10,11]  to  update  the  current  estimate  using  the  last  mea¬ 
sures.  The  resulting  estimator  exhibits  a  stable  error  dynamics  if  the  same  LMI 
relationships  found  for  the  first  estimator  are  satisfied.  In  addition,  the  new  ob¬ 
server  results  in  higher  performance,  as  this  update  provides  a  reduction  of  the 
estimation  error. 

The  paper  is  organized  as  follows.  Section  2  is  devoted  to  the  problem  of 
constructing  a  Luenberger  observer  for  switching  discrete-time  linear  systems, 
with  a  particular  emphasis  on  the  stability  of  the  error  dynamics  and  on  the 
development  of  an  LMI  approach  to  synthesize  such  observers.  In  Section  3, 
a  modified  Luenberger  observer  with  the  related  stability  analysis  is  proposed 
that  enables  one  to  estimate  the  state  of  the  system  using  also  the  last  available 
measures.  Simulation  results  are  illustrated  in  Section  4  to  show  the  performance 
of  the  proposed  estimation  methods.  The  conclusions  are  drawn  in  Section  5. 


2  Switching  Observers  for  Discrete-Time  Linear  Systems 

Consider  the  discrete-time  linear  system 


f  +  l  yLt 


(1) 


where  e  is  the  state  vector,  £  RP  is  the  input  vector,  y_^  G 
is  the  measure  vector,  and  a  :  N  — >  {1,2,  ...,A:}  is  a  function  that  maps 
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the  index  time  stage  into  an  index  set  {1,  2, . . . ,  A:}  .  Each  of  the  indices  cor¬ 
responds  to  a  different  model  of  the  system  and  measurement  equations,  i.e., 

^a(t)  €  =  {Ai,  ^2,  ■  •  •  ,  -Bcr(t)  G  H  .  .  .  ,Bk}  ,  C^(t)  ^ 

c  =  {Ci,C2,...,Ck},  where  A  £  R"’"' ,  and  C,  6  R"’>"‘  for 

I  =  1, 2, . . . ,  A: .  We  assume  that  the  matrices  Ci  G  ,  i  =  1, 2, . . . ,  A: ,  are 

of  full  rank  m  <  n  and  the  output  of  the  function  a(-)  is  known  at  time  t. 
Anyway,  it  is  worth  noting  that  the  matrices  Ba{t)  and  with  time- varying 

dimensions  in  the  number  of  columns  and  rows  (i.e.,  of  p  and  m,  respectively) 
are  allowed.  A  switching  observer  for  (1)  is  the  following; 

—t  {U.t  “  —t)  ’  t  =  0, 1, . . .  (2) 

where  —  to  chosen  “a  priori”  and  L^(t)  is  the  observer  gain  at  the  time 

t ,  £  £.  =  {Li,L2,...,Lk},  and  Li  €  R"^™ ,  i  =  1,2,. . .  ,k .  A  pictorial 

representation  of  such  an  observer  is  shown  in  Fig.  1. 


Fig.  1.  Scheme  of  a  switching  observer. 


Note  that  these  gains  may  change  in  such  a  way  that  the  dimension  m  will 
vary  over  time  due,  for  example,  to  a  variable  number  of  available  measures  at 

time  t.  The  dynamics  of  the  estimation  error  (i.e.,  ~  ^t)  behaves  like  a 

switching  dynamic  system,  thus  a  common  Lyapunov  function  is  searched  for  to 
ensure  stability.  Now,  we  can  state  the  following  theorem. 


Theorem  1.  Consider  the  system  (1)  and  assume  that  the  pairs  [Ai,Ci) ,  i  = 
1, 2, . . . ,  A:,  are  observable.  If  there  exists  a  symmetric  positive  definite  matrix  P 
as  the  solution  of  the  algebraic  Lyapunov  inequalities 

{Ai-LiCifP{Ai-LiCi)-P<0  ,  i  =  l,2,...,k  (3) 

then  the  observer  (2)  involves  an  estimation  error  asymptotically  convergent  to 
zero. 


□ 
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Proof  of  Theorem  1.  Let  us  prove  the  result  stated  in  Theorem  1.  The 
error  dynamics  may  be  computed  by  means  of  equations  (1)  and  (2) 

==  (A^(t)  -  t  — 

If  we  consider  the  Lyapunov  function  Vt  =  ef  ,  where  P  is  a  symmetric 
positive  definite  matrix,  we  obtain  Vt^i  <  Vt ,  Ve^  G  ,  if 

P  (^a(t)  —  ^<T(t)  —  P  <0  ,  t  =  0,  1,  .  ,  .  , 

and  then  (3)  may  be  easily  derived. 


It  is  important  to  recall  that  the  assumption  on  the  observabilty  of  the  pairs 
,  is  necessary  to  guarantee  that  each  inequality  in  (3)  may 
admit  a  solution  for  a  given  positive  definite  matrix  P ,  but  the  existence  of  a 
common  P  satisfying  all  the  inequalities  is  required  to  ensure  stability.  As  it  is 
difficult  to  find  a  common  Lyapunov  function  once  the  gains  i  =  1,2,. . . 
have  been  selected,  we  will  try  to  find  the  gains  and  the  positive  definite  matrix 
P  simultaneously.  Thus,  the  goal  is  to  solve  the  following  problem. 

Problem  1.  Find  Li ,  i  =  1,2, . . .  ,k,  such  that  there  exists  a  symmetric  positive 
definite  matrix  P  solving  the  Lyapunov  inequalities 

{Ai-LfCif  P{Ai-LiCi)-P<0  ,  i  =  .  (4) 


□ 


The  above  problem  may  be  reduced  to  a  simpler  form  that  is  well-suited 
to  being  solved  by  an  LMI  method.  To  this  end,  let  us  consider  the  following 
lemma. 


Lemma  1.  Given  a  symmetric  positive  definite  matrix  P ,  an  inequality 

{Ai-LiCifP{Ai-LiCi)-P<0  (5) 

is  equivalent  to 


(  P  PAi-Y,Ci\ 
\{PAi-YiCif  P  ) 


>  0 


where  Li  =  P  ^Yi,  i  =  l,2,...,k. 


(6) 


□ 


Proof  of  Lemma  1.  Let  us  recall  the  well-known  Schur  complement 
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where  R,  5 ,  and  Q  are  matrices  of  appropriate  dimensions.  If  we  apply  this 
result  by  taking  Q  —  P  ^  S  =  P  Ai  —  YiCi  ^  and  R  =  P ,  we  can  easily  verily 
that  (6)  gives  (5)  if  Li  =  P~^  Yi . 


To  sum  up,  the  solution  of  Problem  1  can  be  obtained  by  solving  the  following 
LMI  problem. 


Problem  2.  Find  P  >  0  and  Yi ,  i  =  1,2, . . .  ,k,  such  that 
/  P  PAi-YiCi\ 

[(.PA-YMf  P  . ‘  ■  <'> 

and  take  the  observer  gains  Li  =  P~  ^  Yi . 


□ 

Problem  2  is  simpler  than  Problem  1,  as  the  former  is  linear  in  the  unknown 
parameters,  whereas  the  latter  is  quadratic  at  the  first  glance.  Moreover,  the 
formulation  of  Problem  2  fits  the  so-called  LMI  framework  [12],  which  enables 
one  to  solve  it  by  means  of  convex  programming  algorithms.  Efficient  numerical 
methods  for  convex  optimization  are  available,  and  the  reader  is  referred  to  [13] 
for  an  introduction  on  this  subject. 

3  An  Enhanced  Projection-Based  Luenberger  Observer 

The  Luenberger  observer  (2)  provides  an  estimate  of  the  state  at  time  t  +  1 
using  the  measures  available  at  time  t  by  means  of  .  As  a  matter  of  fact,  we 
aim  at  determining  the  estimate  using  also  like  a  standard  Kalman 

filter.  To  this  end,  a  method  is  proposed  and  consists  in  updating  the  estimate 
given  by  the  Luenberger  observer  (2)  by  means  of  the  projection  method  [10,11], 
which  allows  one  to  take  into  account  the  last  measures.  More  specifically,  this 
estimation  method  is  performed  as  follows: 

J  1  =  Lt  "k  '®cr(t)  "k  {y.t  ~  ^ar{t)  )  5  i  =  0,  1,  .  .  . 

I +  p  (c'^(t+i)P  {yt+1  - Lt+i) 

where  Xq  =  Xq  is  chosen  “a  priori” ,  is  the  observer  gain  at  time  t , 

i.e.,  e  C  =  {Li,  L2, . . . ,  Tfc} ,  and  P  is  a  positive  definite  matrix.  The 

update  that  enables  one  to  derive  the  new  estimate  using  and  the 

last  measure  vector  is  based  on  a  simple  geometrical  idea  we  will  illustrate 
in  the  following. 

For  the  sake  of  notational  simplicity,  let  assume  x^_^y  is  measured  at  time 
t-fl  by  means  of  =  C x^_^^  (i.e.,  C  is  used  instead  of  ).  Moreover, 
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we  regard  as  an  “a  priori”  estimate  of  at  time  t  +  1  and  want  to 

determine  a  new  estimate  such  that  ||e^i||  <  ||§t4-i||,  where  = 

~^t+i  •  The  state  space  can  be  decomposed  into  two  orthogonal  subspaces, 
like,  for  example,  the  null  space  of  C  (i.e.,  N  (C)  =  {xeM’^:Cx  =  0})  and  its 

orthogonal  space  N  using  the  scalar  product  <  x,z  >p=  x^  Pz,  x,  z  € 

(this  scalar  product  is  well-defined  as  the  matrix  P  is  positive  definite). 
If  P  is  taken  equal  to  the  identity  matrix,  it  is  easy  to  verify  that  N  (C)^  is 
i?(C'^)  (i.e.,  the  space  spanned  by  linear  combinations  of  the  columns  of  the 
matrix  ). 


Fig.  2.  Sketch  to  explain  the  projection  method  {C  replaces  ^(^(t+i)). 


The  decomposition  can  be  accomplished  by  means  of  the  subspaces  given  by 
R(P~^  C'^)  and  its  orthogonal  complement,  instead  of  N  {C)^  and  N  (C) . 
The  reason  for  using  this  subspace  decomposition  concerns  the  stability  of 
the  estimation  error  as  it  will  be  clarified  in  the  following.  Fig.  2  pro¬ 
vides  a  meaningful  geometrical  interpretation  of  the  projection  method  and 
enables  one  to  illustrate  the  rationale  for  the  proposed  approach.  As  can 
be  noticed  in  Fig.  2,  the  projection  of  on  R{P-^C^)  is  equal  to 

p-ic^  {CP-^C^y^  ,  i.e.,  P-^C^  {CP-^Cf^y^  .  Note  that  the 

projection  matrix  P~^C^  [CP~^C'^)  ^  is  well-defined  as  the  matrices  Ci  € 
l^mxTi  ^  z  =  1,2,  ...,/c  (i.e.,  C  in  this  case)  have  been  assumed  of  full  rank 
m  <  n .  In  practice,  the  estimate  of  x^_|.i  is  obtained  by  projecting  on  the 
subspace  corresponding  to  the  new  measure  ,  which  provides  a  new  esti¬ 
mate  x^j  such  that  the  corresponding  estimation  error  is  smaller  than 
that  of  the  previous  error,  i.e.,  ||e^i||  <  ||^t+i||  •  A  pictorial  representation  of 
the  observer  (8)  is  shown  in  Fig.  3. 

As  far  as  it  concerns  the  stability  of  estimation  error  associated  to  (8),  we 
can  state  the  following  result. 
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Fig.  3.  Scheme  of  a  switching  projection-based  observer. 


Theorem  2.  Consider  the  system  (1)  and  assume  that  the  pairs  {Ai,Ci) ,  i  = 
1, 2, . . . ,  /C;  are  observable.  If  there  exists  a  symmetric  positive  definite  matrix  P 
as  the  solution  of  the  algebraic  Lyapunov  inequalities 

(Ai-LiCif  P(Ai-LiCi)-P<0  ,  i  =  l,2,...,k  (9) 

then  the  estimator  (8)  involves  an  estimation  error  asymptotically  convergent  to 
zero. 

□ 


Proof  of  Theorem  2.  The  estimation  error  before  the  projection  update  is 
given  by 

=  «  =  .  (10) 

As  a  consequence  of  the  update  based  on  the  measure  ,  the  estimation 

error  becomes  e+^i  =  /  -  Ca(t+i)  &+i  • 

In  order  to  prove  that  the  resulting  estimator  is  stable,  consider  the  Lyapunov 
functions  Vt  =  ^  P§Lt  P^t  ?  where  P  is  a  symmetric  positive 

definite  matrix.  The  goal  is  to  demonstrate  that  the  estimation  error  ef  con¬ 
verges  asymptotically  to  zero  by  proving  that  is  decreasing  in  t,  Ve^  G  . 
To  this  end,  it  is  sufficient  to  demonstrate  that  VtXi  <  Vt+i ,  t  =  0, 1, . . .  as, 
from  (10),  it  is  obvious  that  Vt+i  <  ,  Ve^  G  if  the  Lyapunov  inequalities 

(9)  are  satisfied  (see  also  the  proof  of  Theorem  1).  Thus,  let  us  consider 

_ j  rp 

^t+1  =  e^+i-P  ^  (^a(t+i) -P  j  P  §.t+i 


-1 
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and  we  can  conclude  since  (^,.(1+1)  CJci+i))  *  ^^((+1)  Sj+i  <  0 , 

Vet+i  G  R" . 


The  design  of  the  observer  (8)  can  be  accomplished  by  solving  the  related 
Problem  2  in  order  to  satisfy  (9)  as  the  requirements  to  apply  the  projection 
update  are  only  that  P  is  a  positive  definite  matrix  and  the  matrices  Q  e 
j^mxn  ^  i  =  1^  2, . . . ,  ,  are  of  full  rank. 

The  projection  method  has  been  successfully  applied  to  the  estimation 
of  a  class  of  continuous-time  nonlinear  systems  with  asynchronous  measure¬ 
ments  [14] .  Moreover,  the  performance  improvements  provided  by  the  projection 
method  will  be  highlighted  by  means  of  the  simulation  results  presented  in  the 
next  section. 


4  Simulation  Results 

In  order  to  show  the  effectiveness  of  the  proposed  estimation  methods  and  the 
feasibility  of  the  related  LMI  design  procedure,  let  us  consider  the  simple  me¬ 
chanical  system  depicted  in  Fig.  4. 


Fig.  4.  Simple  mechanical  system. 


The  continuous-time 
tion: 


dynamics  of  the  system  is  given  by  the  following  equa- 


f  x{t)  ~  Ax{t)  +  B  u{t) 
=  x(t) 


(11) 


where  x{t)  =  [xi{t),X2{t),X3{t),X4{t)f  6  R”*  is  the  state  vector  and  u{t)  e  R 
is  the  scalar  input.  More  specifically,  a;i(t)  is  the  position  and  of  the  mass  mi , 
X2{t)  is  the  speed  of  mi ,  X3{t)  is  the  position  of  the  mass  m2 ,  and  14(4)  is 
the  speed  of  m2  .  The  matrices  A  and  B  are  as  follows; 


0 

1  0 

1  ^ 

/  0  \ 

-{ki  +k2)/mi  -bifmi  —k^jmx 

0 

Bt 

1/mi 

0 

0  0 

1  , 

0 

k^jra^ 

0 

-l>2/m2  j 

0  y 
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where  the  parameters  are  ki  =  0.0642  k2  —  0.1925  ,  mi  = 

5.1962  m2  =  1.7321  bi  —  0.8660  i^p/s,  and  62  =  0.1732i^5i/s . 
The  system  is  switching  in  that  the  matrix  can  assume  values  in  the  set 

{Ci,C2,C3,C4}  with  Cl  =(1,0, 0,0,),  C2  =  (0,1, 0,0),  C3  =  (0,0, 1,0),  and 
C4  =  (0, 0, 0, 1) .  The  choice  among  the  four  candidate  measurement  equations  is 
random,  with  the  same  probability  of  occurrence.  The  system  is  observable  with 
any  of  the  four  measurement  equations.  The  input  u{t)  was  taken  to  be  equal  to 
a  sinusoidal  force,  i.e.,  u{t)  =  k^  sin  (wt) ,  where,  for  each  simulation  run,  ku 
and  w  were  randomly  chosen  in  [0.0, 4.0]  N  and  [0.1, 0.6]  rad/s,  respectively. 
Moreover,  the  initial  states  were  randomly  Gaussian  distributed  around  0  with 
standard  deviations  5.0, 2.0, 5.0,  and  2.0  for  xi,X2,X3,  and  X4,  respectively. 

A  corresponding  discrete-time  model  was  obtained  for  the  same  system  dis¬ 
cretizing  equation  (11)  by  means  of  a  simple  Euleur’s  approximation  with  a  time 
step  equal  to  0.1s.  In  the  discrete-time  setting,  the  standard  routines  of  the 
Matlab  LMI  Control  Toolbox  [15]  provided  the  following  solution  to  Problem  2: 


/  0.1331  0.1175  -0.0873  -0.0159  \ 


1  0.1175  1.4445  -0.0371  -0.1386 

-0.0873  -0.0371  0.0960  0.0351 

-0.0159  -0.1386  0.0351  0.8227 


/  1.0028  \ 

/-1.5816\ 

0.6846  \ 

-0.0623 

1.0246 

-0.0325 

0.9016 

“  -1.1822 

L3  = 

1.0065 

\ -0.0280  y 

'  \  0.1874  J 

1^-0.0386/ 

/  0.4669  \ 
0.1131 
-0.6561 
\  0.9940/ 


The  root  mean  square  (RMS)  error  was  considered  as  a  performance  index.  This 
error  for  the  scalar  variable  Xi{t)  with  respect  to  its  estimate  Xi(t)  at  the  time 
t  for  N  different  trials  is  defined  as 


RMSi{t)  = 


(12) 


where  x^  {t)  is  the  value  of  the  variable  in  the  j-th  run,  f  ^  (t)  is  the  estimate  of 
xj(t) ,  and  i  =  1,  2,  3,  and  4.  In  Fig.  5,  the  simulation  results  obtained  with  the 
two  proposed  observers  using  the  above-written  gains  and  with  initial  estimated 
state  equal  to  0  are  shown  as  far  as  it  regards  the  RMS  estimation  error  on 
500  trials  (i.e.,  N  =  500)  with  different  choices  of  k^  ,  w ,  initial  state  vectors, 
and  switching  sequences.  As  can  be  noticed  in  Fig.  5,  the  enhanced  Luenberger 
observer  exhibits  a  faster  convergence  rate.  The  trajectories  of  the  true  and 
estimated  state  variables  for  a  single  random-chosen  simulation  run  are  shown 
in  Fig.  6. 


Position  [m]  Position  [m]  Position  [m]  Position  [m] 
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RMS  error  on  the  estimate  of  x1 


RMS  error  on  the  estimate  of  x2 


RMS  error  on  the  estimate  of  x3 


Time  [s) 


RMS  error  on  the  estimate  of  x4 


Fig.  5.  RMS  estimation  errors  of  the  switching  observers. 


State  variable  x^  State  variable  ^ 


State  variable 


Time  [s] 


Slate  variable  x^ 


Time  [s] 


Fig.  6.  True  values  and  estimates  of  the  switching  observers  for  a  single  realization. 
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5  Conclusions 

In  this  paper,  estimation  for  a  class  of  hybrid  systems  has  been  considered.  First, 
we  have  addressed  the  problem  of  designing  a  Luenberger  observer  for  a  class  of 
switching  discrete-time  linear  systems.  Conditions  ensuring  the  stability  of  the 
error  dynamics  for  such  an  estimator  have  been  found  and  an  LMI  formulation 
has  been  presented  to  synthesize  the  gains  in  a  straightforward,  efficient  way. 
Second,  an  enhanced  Luenberger  observer  has  been  proposed  to  perform  esti¬ 
mation  using  also  the  last  available  measures.  The  stability  of  the  estimation 
error  for  this  modified  Luenberger  observer  has  been  proved  under  conditions 
that  can  be  ensured  by  solving  the  same  LMI  problem  of  the  first  estimator.  The 
simulation  results  obtained  with  such  observers  for  a  simple  mechanical  system 
show  both  that,  as  expected,  the  proposed  estimators  are  stable  and  that  the 
enhanced  Luenberger  observer  results  in  higher  performance. 

Future  work  will  concern  the  application  of  the  proposed  approach  to  real 
cases  (see  [14]),  where  conventional  estimation  methods  based  on  Kalman  filter¬ 
ing  may  perform  poorly.  Moreover,  further  theoretical  investigations  will  regard 
the  extension  of  the  switching  observer  to  a  more  general  framework,  e.g.,  with 
noises  acting  on  the  system  and  measurement  equations  and  nonlinearities  af¬ 
fecting  the  dynamics. 
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Abstract.  In  a  biological  cell,  cellular  functions  and  the  genetic  regula¬ 
tory  apparatus  are  implemented  and  controlled  by  a  network  of  chemical 
reactions  in  which  regulatory  proteins  can  control  genes  that  produce 
other  regulators,  which  in  turn  control  other  genes.  Further,  the  feed¬ 
back  pathways  appear  to  incorporate  switches  that  result  in  changes  in 
the  dynamic  behavior  of  the  cell.  This  paper  describes  a  hybrid  systems 
approach  to  modeling  the  intra-cellular  network  using  continuous  differ¬ 
ential  equations  to  model  the  feedback  mechanisms  and  mode-switching 
to  describe  the  changes  in  the  underlying  dynamics.  We  use  two  case 
studies  to  illustrate  a  modular  approach  to  modeling  such  networks  and 
describe  the  architectural  and  behavioral  hierarchy  in  the  underlying 
models.  We  describe  these  models  using  Charon  [2],  a  language  that 
allows  formal  description  of  hybrid  systems.  We  provide  preliminary  sim¬ 
ulation  results  that  demonstrate  how  our  approach  can  help  biologists 
in  their  analysis  of  noisy  genetic  circuits.  Finally  we  describe  our  agenda 
for  future  work  that  includes  the  development  of  models  and  simulation 
for  stochastic  hybrid  systems.^ 


1  Introduction 

In  order  to  survive,  organisms  continuously  monitor  their  surroundings  and,  if 
necessary,  adjust  traffic  through  simple  or  complex  combinations  of  genetic  and 
metabolic  networks  to  respond  to  alterations  in  local  conditions.  Local  condi¬ 
tions  include  both  the  physical  environment,  for  example,  temperature  (the  heat 
and  cold  shock  response),  nutrient  and  energy  source  concentrations  (the  strin¬ 
gent  response),  light  (circadian  rhythms),  cell  density  (quorum  sensing  response) 
as  well  as  the  molecular  environment  of  individual  regulatory  components.  Ex¬ 
amples  of  the  latter  include  intracellular  concentrations  of  transcription  factors 
and  allosteric  effectors.  The  availability  of  complete  genomic  information  for  a 
wide  variety  of  organisms  and  the  consequent  attention  on  proteomics  has  dra¬ 
matically  increased  the  number  of  systems  and  components  of  systems  that  are 
involved  in  these  sensing  and  responding  activities  [4,10].  Understanding  how 
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these  biological  systems  are  integrated  and  regulated  and  how  the  regulation 
may  be  influenced,  possibly  for  therapeutic  purposes,  remains  a  significant  chal¬ 
lenge. 

In  this  paper  we  model  and  simulate  examples  of  genetic  and  metabolic  net¬ 
works  using  a  hybrid  systems  approach  that  combines  concepts  and  tools  from 
control  theory  and  computer  science.  First  we  analyze  a  previously  published 
plasmid-based  genetic  network  that  was  designed  and  synthesized  using  three 
repressor  transcription  factors  where  one  repressor  negatively  regulates  the  pro¬ 
duction  of  a  subsequent  repressor  [7].  Then  we  model  a  biologically  important 
genetic  network  that  controls  the  quorum  sensing  response,  an  adaptive  response 
of  certain  gram  negative  bacteria  to  local  population  density  [13,17].  The  quorum 
sensing  response  controls  the  luminescent  behavior  in  certain  strains  of  Vibrio 
which  has  been  linked  to  the  normal  development  of  the  bacterial  host  [18]  as 
well  as  to  medically  important  phenomena  such  as  biofilm  formation  by  Pseu¬ 
domonas  aerugenosa,  an  organism  that  can  cause  overwhelming  pneumonia  and 
septic  shock  [11,20]. 

2  Modeling 

The  genetic  circuits  and  biomolecular  networks  considered  here  and  elsewhere 
are  remarkably  similar  to  hybrid  systems  encountered  in  engineering,  for  exam¬ 
ple  embedded  systems.  In  particular,  it  is  worth  noting  the  following  three  key 
features: 

Concurrency  and  communication.  At  the  intra-cellular  level,  proteins  and 
mRNAs  are  agents  communicating  with  each  other  and  influencing  each 
other’s  behavior.  At  the  inter-cellular  level,  cells  can  be  viewed  as  networked 
agents  interacting  with  each  other  via  different  communication  mechanisms. 
Discrete  and  continuous  behaviors.  At  the  lowest  level,  the  evolution  of  en¬ 
tities  such  as  proteins  can  be  described  by  differential  equations.  Discreteness 
arises  in  two  ways.  First,  a  certain  activity  may  be  triggered  only  when  the 
concentration  of  enabling  quantities  is  above  the  desired  threshold.  This  leads 
to  discrete  switching  between  active  and  dormant  states.  Second,  different 
models  may  be  appropriate  at  different  levels  of  concentration. 

Stochastic  behavior.  Evolution  of  entities  is  not  deterministic,  and  is  better 
captured  by  stochastic  models  that  allow  for  uncertainty  and  noise. 

These  characteristics  are  typical  of  high-level  models  of  embedded  software  such 
as  autonomous  communicating  mobile  robots.  For  describing  such  systems,  we 
have  developed  the  language  CHARON  [2]  which  incorporates  ideas  from  con¬ 
currency  theory  (languages  such  as  CSP  [12]),  object-oriented  software  design 
notations  (such  as  Statecharts  [9]  and  UML  [3]),  and  formal  models  for  hybrid 
systems  (such  as  hybrid  automata  [1]  and  hybrid  I/O  automata  [15]).  The  key 
features  of  CHARON  are: 

Architectural  hierarchy.  The  building  block  for  describing  the  system  ar¬ 
chitecture  is  an  agent  that  communicates  with  its  environment  via  shared 
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variables.  The  language  supports  the  operations  of  composition  of  agents  to 
model  concurrency,  hiding  of  variables  to  restrict  sharing  of  information,  and 
instantiation  of  agents  to  support  reuse. 

Behavior  hierarchy.  The  building  block  for  describing  flow  of  control  inside  an 
atomic  agent  is  a  mode.  A  mode  is  basically  a  hierarchical  state  machine,  that 
is,  a  mode  can  have  submodes  and  transitions  connecting  them.  Variables 
can  be  declared  locally  inside  any  mode  with  standard  scoping  rules  for 
visibility.  Modes  can  be  connected  to  each  other  only  via  well-defined  entry 
and  exit  points.  We  allow  sharing  of  modes  so  that  the  same  mode  definition 
can  be  instantiated  in  multiple  contexts.  Finally,  to  support  exceptions.,  the 
language  allows  group  transitions  from  default  exit  points  that  are  applicable 
to  all  enclosing  modes. 

Discrete  updates.  Discrete  updates  are  specified  by  guarded  actions  labeling 
transitions  connecting  the  modes.  Actions  can  have  calls  to  externally  defined 
Java  functions  which  can  be  used  to  write  complex  data  manipulations.  It 
also  allows  us  to  mimic  stochastic  aspects  through  randomization. 
Continuous  updates.  Some  of  the  variables  in  Charon  can  be  declared  ana¬ 
log,  and  they  flow  continuously  during  continuous  updates  that  model  pas¬ 
sage  of  time.  The  evolution  of  analog  variables  can  be  constrained  in  three 
ways:  differential  constraints  (e.g.  by  equations  such  as  i  =  f(x,u)),  alge¬ 
braic  constraints  (e.g.  by  equations  such  asy  =  g{x,  u)),  and  invariants  (e.g. 

"  2/|  ^  which  limit  the  allowed  durations  of  flows.  Such  constraints  can 
be  declared  at  different  levels  of  the  mode  hierarchy. 

Modular  features  of  CHARON  allow  succinct  and  structured  description  of 
complex  systems.  Similar  features  are  supported  by  the  languages  SHIFT  [6]  and 
Stateflow  (see  www.mathworks.com).  In  CHARON,  modularity  is  not  only  ap¬ 
parent  in  syntax,  but  we  are  developing  analysis  tools  (such  as  simulation)  that 
exploit  this  modularity.  Furthermore,  CHARON  has  formal  foundations  support¬ 
ing  compositional  refinement  calculus  which  allows  relating  different  models  of 
the  system  in  mathematically  precise  manner.  A  formal  mathematical  descrip¬ 
tion  allows  us  to  develop  tools  for  computing  equilibria,  for  reachability  analysis 
and  for  analyzing  properties  like  stability  and  reachability. 

In  the  next  two  sections,  we  will  briefly  describe  case  studies  that  we  have 
used  to  investigate  the  hybrid  systems  approach  to  modeling  biological  systems, 
and  the  applications  of  CHARON  to  these  systems.  We  will  also  illustrate  our 
approach  by  providing  preliminary  simulation  results. 


3  A  Repressilator  Network 

As  noted  in  [5],  most  biomolecular  systems  of  interest  involve  many  interactions 
connected  through  positive  and  negative  feedback  loops  and  an  understanding  of 
their  dynamics  is  hard  to  obtain.  In  this  section  we  will  describe  the  modeling  of 
a  specific  biomolecular  network.  We  will  model  a  repressilator  system  described 
in  [7] .  First  we  provide  some  biological  background  information  and  describe  the 
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protein  network  used  in  [7],  and  then  describe  the  models  of  the  protein  network, 
including  examples  of  Charon  models.^ 


3.1  The  Basic  Phenomena 

In  the  synthetic  oscillatory  network  described  in  [7],  networks  of  interacting 
biomolecules  carry  out  many  essential  functions  in  living  cells.  But  the  design 
principles  of  the  functioning  of  such  networks  still  remain  poorly  understood- 
even  in  relatively  simple  systems  [14].  The  authors  proposed  the  design  and 
construction  of  a  synthetic  protein  network  implementing  a  particular  function. 
Their  motivation  is  that  such  “rational  network  design”  may  lead  to  the  engi¬ 
neering  of  new  cellular  behaviors  and  to  improved  understanding  of  naturally 
occuring  networks. 

The  repressilator  system  described  in  [7]  contains  three  proteins,  namely 
lad,  tetR,  and  cl.  The  protein  lad  represses  the  protein  tetR,  tetR  represses  cl, 
whereas  cl  represses  lad,  thus  completing  a  feedback  system  called  a  repressi¬ 
lator  system.  The  dynamics  of  the  network  depend  on  the  transcription  rates, 
translation  rates,  and  decay  rates  of  proteins  and  messenger  RNAs.  Depending 
on  the  values  of  the  different  parameters  in  the  model,  the  system  might  converge 
to  a  stable  limit  cycle  or  become  unstable. 

3.2  Approaches  to  Modeling 

It  is  well  known  in  mechanics  and  thermodynamics  that  there  are  two  different 
approaches  to  modeling  systems  such  as  the  repressilator  system.  At  reasonably 
high  molecular  concentrations,  one  can  adopt  continuum  models  which  lend 
themselves  to  deterministic  models  involving  ordinary  and  partial  differential 
equations.  At  lower  concentrations,  the  discrete  molecular  interactions  become 
important  and  deterministic  models  are  difficult  to  obtain  [8]. 


The  Deterministic,  Continuous  Approximation.  We  will  consider  the 
three  repressor  protein  concentrations  e  P  =  {lad, tetR, cl}  and  their  cor¬ 
responding  mRNA  concentrations  6  P  as  continuous  dynamic  variables. 
The  system  kinetics  are  determined  by  the  following  six  coupled  first-order  dif¬ 
ferential  equations. 


drrii  a 

^  =  -/3(p,-m0 


{i,i)  e  {(lacl,cl),  (tetR,lacI),  (cI,tetR)} 

^  For  more  information  on  CHARON  or  sample  Charon  code,  please  check 
http://www.cis.upenn.edu/mobies/charon/  or  contact  ivancic@seas.upenn.edu. 
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The  equations  use  various  constants.  The  leakiness  of  the  promoter  a  is  the 
number  of  protein  copies  per  cell  produced  from  a  given  promoter  type  during 
continuous  growth  in  the  presence  of  saturating  repressor  amounts.  During  the 
absence  of  the  repressor,  we  have  a  +  Oq  number  of  protein  copies  per  cell.  The 
ratio  of  the  protein  decay  rate  to  the  mRNA  decay  rate  is  denoted  by  /?,  while 
n  stands  for  the  so  called  Hill  coefficient. 

The  Stochastic,  Discrete  Approximation.  The  continuous  analysis  neglects 
the  discrete  nature  of  molecular  components  and  the  stochastic  character  of  their 
interaction  [7].  Following  [7],  we  adopt  the  stochastic  approximation  as  described 
by  Gillespie  in  [8].  The  various  proteins  and  mRNAs  are  modeled  by  discrete 
variables  corresponding  to  the  number  of  molecules  measuring  concentration, 
and  are  updated  at  discrete  time  intervals  by  stochastic  rules. 

3.3  Charon  Model 

In  this  section  we  will  present  the  repressilator  system  models  as  described  in  [7] 
using  the  Charon  language.  We  will  present  many  of  the  advantages  that  the 
Charon  language  has  to  offer  for  modeling  such  biomoiecular  models. 

Our  model  will  define  a  generic  protein  model  as  an  agent  in  Charon.  We  will 
instantiate  this  agent  model  to  obtain  the  three  proteins  lad,  tetR,  and  cl.  The 
approximation  models  will  be  implemented  inside  the  modes  of  the  protein  agent. 
To  present  another  feature  of  our  language,  we  will  also  describe  a  combination 
of  the  discrete  and  the  continuous  model  into  one  modeling  system. 

The  Protein  Agent  in  the  Continuous  Approximation.  In  this  section 
we  will  describe  a  Charon  model  of  a  generic  protein  agent.  We  have  a  con¬ 
tinuous  input  variable  which  represents  the  repressor  protein  concentration  pr. 
This  means,  that  the  environment  of  this  protein  agent  supplies  the  value  of  this 
variable,  and  it  cannot  be  changed  by  the  protein  agent.  The  protein  agent  has  a 
continuous  private  variable  representing  the  messenger  RNA  concentration.  Pri¬ 
vate  variables  cannot  be  seen  outside  the  agent  and  can  be  updated  internally  for 
internal  use  only.  The  output  of  the  protein  agent  is  a  continuous  variable  rep¬ 
resenting  the  protein  concentration.  Output  variables  are  updated  by  the  agent, 
and  can  be  used  as  input  variables  to  other  agents.  The  generic  protein  agent 
has  parameters  ao,  a, /?,  n,po?  ^iRd  mo-  By  instantiating  these  parameters  with 
values,  we  can  obtain  instantiated  protein  agents  representing  a  specific  protein. 
The  parameters  po  and  mo  will  be  used  for  initialization  purposes  and  stand  for 
the  initial  protein  concentration  and  the  initial  messenger  RNA  concentration 
respectively.  The  following  represents  the  corresponding  Charon  code. 

agent  contProtein  (real  pO  ,  mO  ,  alphaO  ,  alpha  ,  beta  ,  n){ 
write  cinalog  real  p  -  pO  ;  //protein  concentration 
read  analog  real  pR  ;  //repressor  protein  concentration 
private  analog  real  m  =  mO  ;  //messenger  RNA  concentration 
mode  cont  =  continuous  (  alphaO  ,  alpha  ,  beta  ,  n  )  ;  } 
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Fig.  1.  A  generic  protein  agent  for  the  continuous  approximation  model 


We  still  need  to  define  the  behavior  of  the  agent.  The  behavior  is  described 
by  the  modes  of  the  agent.  The  behavior  of  the  generic  protein  agent  is  defined 
in  cont,  which  is  an  instantiation  of  a  generic  continuous  mode  defined  by  the 
following  code.  A  graphical  version  of  the  generic  protein  model  can  be  found  in 
Figure  1. 

mode  continuous  (real  alphaO  ,  alpha  ,  beta  ,  n){ 
write  cinalog  real  p  ;  //protein  concentration 
read  analog  real  pR  ;  //repressor  protein  concentration 
private  analog  real  m  ;  //messenger  RNA  concentration 
diff  fflRNA  {  d(m)  =  -m  +  alpha  /  (l+pR"n)  +  alphaO  } 
diff  proteinConcentration  ■[  d(p)  =  -beta  *  (p-m)  >  } 


Fig.  2.  Composed  repressilator  system  using  the  instantiated  generic  protein  agent 


Instantiation  and  Concurrency.  We  defined  a  generic  protein  agent  in  the 
previous  section.  We  have  to  instantiate  this  generic  agent  model  to  get  the 
three  proteins  used  in  the  system.  We  also  want  the  three  proteins  lad,  tetR, 
and  cl  to  run  in  parallel  and  to  influence  each  other.  Notice  the  use  of  renaming 
of  variables  to  couple  the  three  instantiated  protein  agents  to  influence  each 
other.  A  graphical  version  of  the  composed  system  is  illustrated  in  Figure  2.  The 
following  represents  the  corresponding  Charon  code  using  some  values  for  the 
parameters.  A  simulation  trace  generated  by  the  Charon  tool-set  is  given  in 
Figure  3. 
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agent  RepressilatorSystem  (){ 

private  analog  real  pi  ,  p2  ,  p3  ; 

agent  lad  =  contProtein  (  ...  )  [  p  ,  pR 

agent  tetR  =  contProtein  (  ...  )  [  p  ,  pR 

agent  cl  =  contProtein  (  ...  )  [  p  ,  pR 


pi  ,  p3  ]  ; 

p2  ,  pi  ]  ; 

p3  ,  p2  ]  ;  } 


Fig.  3.  Simulation  trace  for  the  repressilator  system  showing  stable  oscillations  for  the 
three  protein  concentration  pi ,  P2 ,  P3  over  time. 


The  Protein  Agent  in  the  Discrete  Approximation.  In  this  section  we 
will  present  a  possible  model  for  a  discrete  approximation  of  a  protein  agent.  As 
we  did  it  for  the  continuous  case,  we  will  again  define  a  generic  protein  agent, 
that  can  be  instantiated  to  build  a  system  of  proteins.  Our  model  works  as 
follows.  We  have  an  integer  variable  n  that  keeps  track  of  the  number  of  protein 
molecules  which  is  the  output  of  the  agent.  The  input  to  the  agent  is  the  number 
of  repressor  protein  molecules  hr.  Depending  on  various  parameters,  we  want 
to  increase  or  decrease  the  number  of  protein  molecules  by  one  at  a  time.  The 
basic  idea  is  to  use  stochastic  simulation  as  described  in  [8].  The  parameters 
that  influence  the  stochastic  simulation  are  binding  and  unbinding  of  proteins 
on  two-sided  promoters,  the  protein  and  mRNA  decay  rates,  and  translation. 
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Fig.  4.  A  generic  protein  agent  for  the  combined  framework  using  continuous  and 
discrete  approximation  model 


Combining  the  two  Models  into  one  Framework.  The  two  different  models 
for  the  repressilator  system  can  be  combined  into  one  framework.  The  basic  idea 
is  to  use  the  deterministic  continuous  model  whenever  the  concentration  of  the 
protein  is  high  enough,  whereas  we  would  switch  to  the  discrete,  stochastic  model 
if  the  concentration  would  fall  below  a  certain  threshold  value.  Figure  4  gives  an 
intuitive  graphical  representation  of  the  protein  agent  with  both  the  continuous 
and  discrete  approximation. 


4  Quorum  Sensing  in  Bacteria 

A  good  illustration  of  multicellular  behavior  in  prokaryotes  is  the  cell-density- 
dependent  gene  expression.  In  this  process,  a  single  cell  is  able  to  sense  when 
a  quorum  of  bacteria,  a  minimum  population  unit,  is  achieved.  Under  these 
conditions,  certain  behavior  is  efficiently  performed  by  the  quorum,  such  as  bio¬ 
luminescence,  which  is  the  best  known  model  for  understanding  the  mechanism 
of  cell-density-dependent  gene  expression.  In  this  section,  we  will  describe  a  hy¬ 
brid  system  model  that  captures  the  changes  in  dynamics  of  the  biochemical 
reactions  observed  in  the  literature  [13,16,17]. 


4.1  The  Basic  Phenomena 

Vibrio  fischeri  is  a  marine  bacterium  that  can  be  found  both  as  a  free-living 
organism  and  as  a  symbiont  of  some  marine  fish  and  squid.  As  a  free-living  or¬ 
ganism,  V.  fisheri  exists  at  low  densities  (less  than  500  cells  per  ml  of  seawater) 
and  appears  to  be  non-luminescent.  As  a  symbiont,  the  bacteria  live  at  high 
densities  and  are,  usually,  luminescent.  In  a  liquid  culture,  the  bacteria’s  level  of 
luminescence  is  low  until  the  culture  reaches  mid  to  late  exponential  phase.  A 
dramatic  increase  in  luminescence  is  observed  at  that  time  due  to  the  transcrip¬ 
tional  activation  of  the  lux  genes.  Once  the  bacteria  reach  stationary  phase,  the 
level  of  luminescence  decreases. 

The  /w3;regulon  [17]  contains  two  operons,  Ol  and  Or  (see  Figure  5).  The  left 
operon  Ol  contains  the  luxR  gene  encoding  the  protein  LuxR,  a  transcriptional 
activator  of  the  system.  The  right  operon  Or  contains  seven  genes  luxICDABEG. 
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Fig.  5.  A  portion  of  DNA  emphasizing  luxR  and  luxICDABEG  genes  and  the  binding 
sites  for  LuxR  complex  and  CRP 


Protein  LuxI,  the  product  of  the  luxi  gene  is  required  for  endogenous  production 
of  autoinducer,  a  small  molecule  capable  of  diffusing  in  and  out  of  the  cell  mem¬ 
brane.  Genes  luxA  and  luxB  encode  two  subunits  of  luciferase.  The  trio  luxC, 
luxD,  and  luxE  code  for  the  subunits  of  a  protein  complex  which  provides  an 
aldehyde  substrate  for  luciferase.  The  function  of  luxG  is  unknown.  The  autoin¬ 
ducer  Ai  binds  to  protein  LuxR  to  form  a  complex  Co.  The  two  operons  are 
separated  by  a  regulatory  region  that  contains  a  binding  site  for  the  cyclic  AMP 
receptor  protein  CRP  and  a  binding  site  for  the  complex  Co. 

The  transcription  of  luxR  is  regulated  by  both  CRP  and  Co.  We  can  distin¬ 
guish  among  the  following  three  different  cases: 

—  Case  Ol-1  In  the  absence  of  the  autoinducer,  CRP  activates  expression 
by  initiating  two  RNA  transcripts. 

—  Case  Ol-2  At  low  autoinducer  concentrations,  luxR  transcription  is  stimu¬ 
lated  by  increasing  CRP-dependent  transcription  and  by  Co-dependent  tran¬ 
scription  from  another  transcriptional  start  site. 

—  Case  Ol“3  At  high  autoinducer  concentrations,  luxR  transcription  is  re¬ 
pressed  through  a  second,  weaker  Co  binding  site  located  in  luxD. 

Likewise,  transcription  of  Or  is  regulated  by  both  CRP  and  Co.  We  distinguish 
two  different  cases: 

—  Case  Or-1  In  the  absence  of  autoinducer,  CRP  represses  Or  transcription. 

—  Case  Or-2  In  the  presence  of  autoinducer,  Co  activates  transcription  of  Or. 

These  cases  will  be  interpreted  as  modes  as  seen  later  in  the  paper. 

4.2  Mathematical  Model 

In  this  section,  we  develop  a  mathematical  model  for  the  luminescence  phe¬ 
nomenon  in  one  bacterium  of  V.  fischeri,  describing  the  concentrations  of  the 
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relevant  mRNA’s,  proteins,  and  small  molecules.  As  described  in  Section  4.1  the 
mechanism  of  transcription  activation  of  both  operons  is  highly  dependent  on 
the  concentration  of  autoinducer,  so  the  time  evolution  of  the  system  cannot  be 
described  by  one  set  of  continuous  differential  equations.^  Combining  cases  for 
Ol  and  Or  given  in  the  previous  section,  yields  three  modes,  which  we  call  OFF, 
POS  and  NEG.  The  transitions  between  modes  are  governed  by  the  level  of  inter¬ 
nal  autoinducer  which  we  represent  by  [Ai].  Mode  OFF  corresponds  to  very  low 
or  zero  concentration  of  autoinducer  ( [Ai]  <  [Ai]  _ )  within  the  bacterium  and  no 
luminescence  is  observed.  The  system  is  in  mode  POS  when  the  concentration 
of  internal  autoinducer  is  low  ([Ai]_  <  [Ai]  <  [Ai]_^).  This  mode  corresponds 
to  positive  growth  and  increasing  concentration  of  autoinducer.  Luminescence 
is  observed,  as  are  higher  concentrations  of  proteins  LuxA,  LuxB,  LuxC,  LuxD, 
and  LuxE.  The  transition  to  mode  NEG  (negative  growth)  occurs  at  high  levels 
of  autoinducer  ([Ai]  >  [Ai]_,_). 

We  use  the  following  rate  equation  to  describe  the  concentration  for  any 
molecular  species  (mRNA,  protein,  protein  complex,  or  small  molecule)  [19]: 
d\x\ 

~  synthesis  -  decay  ±  transformation  ±  transport  (1) 

The  synthesis  term  represents  transcription  for  mRNA  and  translation  for  pro¬ 
teins.  The  decay  term  represents  a  first  order  degradation  process.  The  transfor¬ 
mation  term  describes  reactions  such  as  cleavage  or  ligand-binding  that  do  not 
destroy  the  protein,  but  do  remove  its  ability  to  participate  in  specific  reactions. 
Finally,  molecular  species  may  participate  in  transport  processes,  like  passive 
diffusion  or  active  transport  through  a  membrane. 

The  biomolecular  system  can  be  described  in  a  nine  dimensional  state  space. 
The  nine  variables,  X\,X2, .  ^ describe  the  concentrations  of  different  mole¬ 
cules  as  follows: 

xi  ~  mRNA  transcribed  from  Or, 

X2  =  mRNA  transcribed  from  Or, 
xs  =  protein  LuxR, 

X4  =  protein  LuxI, 

X5  =  protein  LuxA, 
xq  ~  protein  LuxB, 

xj  =  autoinducer  inside  the  bacterium  Ai, 

Xs  =  LuxR:Ai  complex  Co, 
xg  =  autoinducer  outside  the  bacterium  Aig^., 
where  Ai  is  the  dimensionless  version  of  [Ai]. 

For  simplicity,  we  have  assumed  that  the  concentrations  of  CRP  and  of  the 
substrate  necessary  for  endogenous  production  of  Ai  are  constant.  Further,  we 
have  neglected  the  decay  rates  for  chemical  compounds.  Finally,  we  assume  that 


^  In  [13] ,  the  differential  equations  for  the  low  autoinducer  concentration  are  described. 
The  model  presented  here  describes  a  wider  range  of  operating  conditions. 
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the  concentrations  of  LuxC,  LuxD,  and  LuxE  are  similar  to  those  of  Lux  A  and 
LuxB. 

The  (continuous)  differential  equations  for  each  mode  are  of  the  form 
X  =  P{x)  where  x  =  [xi,  X2,  e  IR®,  /“  =  [/j,  f^,  and 

i  €  {OFF,  POS,  NEG}.  The  components  of  the  vector  fields  are  explicitly 
given  by: 


fOFF 

= 

m 

fPOS 

= 

fNEG 

Jl 

-T]iXi 

fOFF 

= 

-r]2^2 

fPOS _ fNEG 

^2  {  1/82  1  ^1^82  ^2  1 

\^82  *^8  / 

fi 

= 

r]3  {xi  -  Xs)  - 

fi 

7^4  (^X2  ^4)  7*4^4 

fi 

7/5  {^2  -  ^5) 

fi 

= 

Ve  (^2  - 

fi 

= 

-r]7X7  +  r4X4^  -  rmem  {^7  -  Xg)  -  r37M^3X7 

fi 

= 

-r]8X8  +  r37^AiX3X7 

fi 

= 

-r]7X9  +  rmem{X7  -  Xg)  F  U 

where,  in  the  last  seven 

equations  /j  is  independent  of  the  mode.  All  the  quan- 

tities  in  the  above  model  are  non-dimensional,  rji  =  To/Hi  where  To  is  the 
characteristic  time  constant  of  the  system  and  Hi  is  the  half-life  (inverse  of  the 
decay  rate)  of  molecule  Xi.  i/ij  is  a  cooperativity  coefficient  while  Kij  describes 
the  potency  of  the  regulation  of  the  transcription  of  mRNA  j  by  protein  i.  r  de¬ 
notes  transformation  and  transfer  rates.  For  example  Tmem  is  the  transfer  rate  of 
autoinducer  through  the  membrane  of  the  cell  while  and  are  transfor¬ 
mation  rates  obtained  by  non-dimensionalizing  the  binding  rate  of  the  reaction 
between  Ai  and  LuxR  in  two  different  ways,  c  is  dependent  on  the  concentration 
of  CRP  and  its  affinity  to  the  corresponding  binding  site,  and,  as  stated  ear¬ 
lier,  is  assumed  to  be  constant.  Finally,  u  emulates  an  external  source  of  Ai  and 
is  used  to  simulate  the  sensitivity  of  the  bacterium  to  changes  of  autoinducer 
concentration  in  the  exterior. 

We  regard  u  as  an  input  to  our  system.  Since  proteins  LuxA  and  LuxB  are 
subunits  of  luciferase,  which  produces  luminescence,  it  is  reasonable  to  assume 
that  the  level  of  luminescence  is  proportional  to  the  product  of  the  concentrations 
of  LuxA  and  LuxB,  which  we  choose  to  be  the  output  of  the  system. 

4.3  Charon  Model 

The  behavioral  hierarchy  in  CHARON  (see  Figure  6)  is  characterized  by  three 
different  behaviors  which  are  represented  by  three  different  modes,  namely  OFF, 
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POS,  and  NEG,  Many  of  the  differential  equations  governing  the  dynamics  of 
the  system  are  shared  between  the  modes.  We  will  introduce  the  notion  of  mode 
hierarchy  to  extract  the  shared  constraints.  Through  the  notion  of  submodes 
and  scoping,  we  can  simplify  the  description  of  the  respective  modes  OFF,  POS, 
and  NEG. 


agent  vibria.lscheri 

=  /.Y^I  i€{OFF,POS,ON },  j  =  3....9 


nuxle  POS-NEG 


iefPOS.NEGJ 


At  >  Ar. 

mode  POS  ' 

S  d 

At  >  At^  J 

'  mMfcNEG 

.  j 

Ai  <  Ar_ 

J  \ 

\  r 

Ai  <  Ai 

Fig.  6.  Charon  structure  of  the  system 


Figure  7  illustrates  the  response  {i.e.,  luminescence)  of  the  bacterium  to  a 
perturbation  in  the  concentration  of  external  autoinducer  that  takes  the  form 
of  a  rectangular  pulse.  The  magnitude  of  the  step  has  been  chosen  to  make 
the  system  go  through  all  three  modes.  The  results  confirm  the  experimental 
observations  [17]:  luminescence  increases  during  mode  POS  and  decreases  in 
mode  NEG]  there  is  no  luminescence  in  mode  OFF.  The  switch  history  and  the 
time  evolution  of  the  concentrations  of  the  significant  molecules  in  the  system 
are  also  shown.  In  mode  OFF,  all  molecules  decay  to  zero,  except  for  mRNA 
Ol  and  the  corresponding  protein  R,  as  expected.  For  a  short  time,  in  mode 
POS,  all  the  concentrations  increase  until  the  internal  autoinducer  reaches  a  high 
concentration,  when  the  system  is  switched  to  mode  NEG.  In  this  last  mode, 
everything  decays  to  zero,  except  for  internal  autoinducer  which  can  reach  a 
stable  non-zero  value  dependent  on  the  size  of  the  step  of  external  autoinducer. 


5  Conclusions 

In  this  paper  we  have  shown  that  biological  cellular  networks  can  be  natu¬ 
rally  modeled  as  hybrid  systems.  In  particular,  the  protein  repressilator  system 
switches  between  a  continuous  deterministic  model  at  high  concentrations,  and 
a  timed,  discrete,  stochastic  model  at  low  concentrations.  Similarly,  the  lumi¬ 
nescence  control  of  Vibrio  fischeri  is  naturally  modeled  as  a  multi-modal  hybrid 
system,  resulting  in  simulations  that  are  in  accordance  with  experimental  obser¬ 
vations.  The  hybrid  nature  of  such  protein  networks  can  be  very  easily  expressed 
and  simulated  in  Charon,  which  may  offer  us  better  and  a  more  global  under¬ 
standing  of  biological  networks. 
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Fig.  7.  Increase  in  external  autoinducer  produces  luminescence;  (a) input  -  external 
source  of  autoinducer;  (b)  switch  history;  (c)  output  (luminescence)-  product  of  con¬ 
centrations  of  proteins  A  and  B]  (d)  and  (e)  time  -  evolution  of  concentrations; 


The  enormous  complexity  of  large  scale  biological  networks  will  present  us 
with  great  challenges  that  we  must  face.  Exploiting  the  structure  of  biological 
systems  will  be  critical  for  scaling  the  applicability  of  the  modeling,  analysis,  and 
simulation  tools.  It  is  therefore  extremely  encouraging  that  the  two  case  studies 
presented  in  this  paper  exhibit  the  architectural  paradigms  of  modern  software 
engineering. 

We  envision  the  link  between  hybrid  systems  technology,  and  biology  to 
strengthen.  The  scalable  nature  of  computational  tools  like  Charon  will  en¬ 
able  the  unified  and  improved  modeling  of  biological  cellular  networks,  leading 
to  better  understanding,  as  well  as  providing  us  with  the  opportunity  to  deter¬ 
mine  how  local  biological  changes  can  affect  global  behavior.  Conversely,  a  good 
understanding  of  the  robustness  of  noisy  biological  networks  will  lead  to  new 
approaches  to  designing  networked  embedded  systems. 

The  case  studies  also  highlight  the  need  for  developing  a  theory  of  stochastic 
hybrid  systems,  for  instance,  for  modeling  rate  equations  of  biochemical  reac¬ 
tions.  We  believe  that  mathematical  and  computational  tools  for  the  analysis 
of  such  systems  present  a  research  challenge  for  the  hybrid  systems  commu¬ 
nity,  while  presenting  a  significant  potential  for  greatly  impacting  post  genomics 
research. 
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Abstract.  In  this  paper,  we  develop  a  theory  of  modular  design  and 
refinement  of  hierarchical  hybrid  systems.  In  particular,  we  present  com¬ 
positional  trace-based  semantics  for  the  language  CHARON  that  allows 
modular  specification  of  interacting  hybrid  systems.  For  hierarchical  de¬ 
scription  of  the  system  architecture,  Charon  supports  building  complex 
agents  via  the  operations  of  instantiation,  hiding,  and  parallel  composi¬ 
tion.  For  hierarchical  description  of  the  behavior  of  atomic  components, 
Charon  supports  building  complex  modes  via  the  operations  of  instan¬ 
tiation,  scoping,  and  encapsulation.  We  develop  an  observational  trace 
semantics  for  agents  as  well  as  for  modes,  and  define  a  notion  of  refine¬ 
ment  for  both,  based  on  trace  inclusion.  We  show  this  semantics  to  be 
compositional  with  respect  to  the  constructs  in  the  language. 

1  Introduction 

Modern  software  design  paradigms  promote  hierarchy  as  one  of  the  key  con¬ 
structs  for  structuring  complex  specifications.  We  are  concerned  with  two  dis¬ 
tinct  notions  of  hierarchy.  In  architectural  hierarchy^  a  system  with  a  collection 
of  communicating  agents  is  constructed  by  parallel  composition  of  atomic  agents, 
and  in  behavioral  hierarchy,  the  behavior  of  an  individual  agent  is  described  by 
hierarchical  sequential  composition.  The  former  hierarchy  is  present  in  almost  all 
concurrency  formalisms,  and  the  latter,  while  present  in  all  block-structured  pro¬ 
gramming  languages,  was  introduced  for  state-machine-based  modeling  in  Stat- 
ECHARTS  [9],  and  forms  an  integral  part  of  modern  notations  such  as  UML  [5]. 

A  hybrid  system  typically  consists  of  a  collection  of  digital  programs  that 
interact  with  each  other  and  with  an  analog  environment.  Specifications  of  hybrid 
systems  integrate  state-machine  models  of  discrete  behavior  with  differential 
equations  for  continuous  behavior.  This  paper  is  about  developing  a  formal  and 
compositional  semantics  of  hierarchical  hybrid  specifications.  Formal  semantics 
leads  to  definitions  of  semantic  equivalence  (or  refinement)  of  specifications  based 
on  their  observable  behaviors,  and  compositionality  means  that  semantics  of  a 
component  can  be  constructed  from  the  semantics  of  its  subcomponents.  Such 
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formal  compositional  semantics  is  a  cornerstone  of  concurrency  frameworks  such 
as  CSP  [11]  and  CCS  [14],  and  is  a  prerequisite  for  developing  modular  reasoning 
principles  such  as  compositional  model  checking  and  systematic  design  principles 
such  as  stepwise  refinement. 

The  main  contribution  of  the  paper  is  a  formal  compositional  semantics  for 
the  language  CHARON  [3]  with  an  accompanying  compositional  refinement  cal¬ 
culus.  The  building  block  for  describing  the  system  architecture  is  an  agent  that 
communicates  with  its  environment  via  shared  variables.  The  language  supports 
the  operations  of  composition  of  agents  to  model  concurrency,  hiding  of  variables 
to  restrict  sharing  of  information,  and  instantiation  of  agents  to  support  reuse. 
The  building  block  for  describing  flow  of  control  inside  an  atomic  agent  is  a 
mode.  A  mode  is  basically  a  hierarchical  state  machine,  that  is,  a  mode  can  have 
submodes  and  transitions  connecting  them.  Variables  can  be  declared  locally  in¬ 
side  any  mode  with  standard  scoping  rules  for  visibility.  Modes  can  be  connected 
to  each  other  only  via  well-defined  entry  and  exit  points.  We  allow  sharing  of 
modes  so  that  the  same  mode  definition  can  be  instantiated  in  multiple  con¬ 
texts.  To  support  exceptions.,  the  language  allows  group  transitions  from  default 
exit  points  that  are  applicable  to  all  enclosing  modes,  and  to  support  history 
retention,  the  language  allows  default  entry  transitions  that  restore  the  local 
state  within  a  mode  from  the  most  recent  exit.  Discrete  updates  are  specified  by 
guarded  actions  labeling  transitions  connecting  the  modes.  Some  of  the  variables 
in  Charon  can  be  declared  analog,  and  they  flow  continuously  during  continu¬ 
ous  updates  that  model  passage  of  time.  The  evolution  of  analog  variables  can 
be  constrained  in  three  ways:  differential  constraints  (e.g.  by  equations  such  as 
X  =  f{x,u)),  algebraic  constraints  (e.g.  by  equations  such  as  y  =  g{x,u)),  and 
invariants  (e.g.  \x  -  y\  <  e)  which  limit  the  allowed  durations  of  flows.  Such 
constraints  can  be  declared  at  different  levels  of  the  mode  hierarchy. 

To  define  the  modular  semantics  for  modes,  with  each  mode  we  associate  two 
relations,  one  capturing  its  discrete  behavior  and  one  capturing  its  continuous 
behavior.  Defining  the  discrete  relation  is  tricky  in  presence  of  features  such 
as  group  transitions,  exceptions,  and  history  retention.  Our  solution  relies  on  a 
closure  construction,  inspired  by  a  similar  construction  for  hierarchical  discrete 
systems  [2],  which  allows  us  to  treat  the  transfer  of  control  between  a  mode  and 
its  environment  as  a  game. 

While  discrete  steps  of  a  mode  and  its  environment  are  interleaved,  continu¬ 
ous  steps  need  to  be  synchronized  as  time  is  a  global  parameter.  In  fact,  during 
a  flow,  all  active  hierarchically  nested  modes  must  participate.  To  allow  flexible 
and  hierarchical  specifications,  in  Charon,  flow  constraints  can  be  specified  at 
all  levels  of  the  hierarchy.  To  formalize  this  feature  in  a  consistent  and  modular 
manner,  we  require  that  a  mode  can  participate  in  a  flow  only  when  the  control 
is  at  its  default  exit  point.  Then,  all  applicable  constraints  are  properly  used  to 
define  permitted  flows. 

The  discrete  and  continuous  relations  of  a  mode  allow  us  to  define  executions 
of  a  mode,  and  corresponding  traces  are  obtained  by  projecting  out  the  private 
variables.  We  show  that  the  set  of  traces  of  a  mode  can  be  constructed  from 
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the  traces  of  its  submodes.  This  compositionality  result  leads  to  a  compositional 
notion  of  refinement  for  modes.  A  mode  M  refines  a  mode  N  if  they  have 
the  same  interface  in  terms  of  entry /exit  points  and  shared  variables,  and  the 
traces  of  M  is  a  subset  of  traces  of  N,  This  notion  admits  modular  reasoning 
in  the  following  manner.  Suppose  we  obtain  an  implementation  design  I  from 
a  specification  design  S  simply  by  locally  replacing  some  submode  AT  in  5  by  a 
submode  M.  Then,  to  show  I  refines  5,  it  suffices  to  show  that  M  refines  N. 
We  illustrate  this  benefit  by  a  simple  example. 

Once  we  have  the  compositionality  results  for  modes,  analogous  results  for 
agents  are  relatively  straightforward.  We  define  an  observational  trace  semantics 
for  agents,  a  resulting  notion  of  refinement,  and  show  it  to  be  compositional  with 
respect  to  the  operations  of  parallel  composition,  hiding,  and  instantiation. 

Related  work.  Early  formal  models  for  hybrid  systems  include  phase  tran¬ 
sition  systems  [13]  and  hybrid  automata  [1].  Models  such  as  hybrid  I/O  au¬ 
tomata  [12]  and  hybrid  modules  [4]  allow  compositional  treatment  of  concurrent 
hybrid  behaviors.  The  notion  of  hierarchical  state  machines  was  introduced  in 
StatechartS  [9],  and  is  present  in  many  software  design  paradigms  such  as 
Uml  [5].  Our  treatment  of  hierarchy  is  closest  to  hierarchical  reactive  mod¬ 
ules  [2]  which  shows  how  to  define  a  modular  semantics  for  hierarchical  (dis¬ 
crete)  modes.  Tools  such  as  SHIFT  [7],  PTOLEMY  [6],  and  Stateflow  (see 
ww.mathworks.com)  allow  hierarchical  specifications  of  hybrid  behavior,  but 
formal  semantics  has  not  been  a  concern.  HyCharts  [8]  presents  a  hierarchical 
model  with  modular  operational  semantics,  but  does  not  consider  refinement. 
Masaccio  [10]  is  a  formal  model  for  hierarchical  hybrid  systems.  While  same  in 
spirit,  it  differs  from  our  model  in  many  technically  significant  aspects:  it  allows 
nesting  of  sequential  and  parallel  composition,  and  allows  a  more  general  form  of 
synchronous  communication,  but  disallows  high-level  features  of  CHARON  modes 
such  as  exceptions,  history  retention,  and  specification  of  constraints  at  various 
levels. 


2  Motivational  Example 

In  this  section,  we  present  a  simple  example  that  outlines  features,  useful  in 
a  specification  language  for  hybrid  systems.  We  also  point  out  the  difficulties 
of  defining  semantics  for  such  a  language.  Then  we  give  the  intuition  for  our 
approach  to  the  semantics  definition,  which  allows  us  to  overcome  the  difficulties. 

Our  example  is  a  system  that  controls  the  level  of  liquid  in  a  leaky  tank. 
The  level  is  controlled  by  infusing  a  flow  of  liquid  into  the  tank.  The  level  in  the 
tank  can  be  measured  directly,  but  the  rate  of  the  leak  has  to  be  estimated.  The 
controller  has  two  goals:  first,  it  must  make  sure  that  the  level  is  within  some 
critical  bounds.  If  it  is  not,  emergency  measures  are  taken  to  make  the  level  safe. 
When  the  level  is  safe,  the  controller  should  change  the  infusion  rate  according 
to  instructions  of  the  user.  To  do  that,  the  controller  periodically  recomputes 
the  desired  rate  of  change  for  infusion  and  maintains  the  computed  rate  until 
the  next  update. 
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We  now  present  a  hierarchical  description  of  the  system  in  Charon.  The 
hierarchy  in  CHARON  is  twofold.  The  architectural  hierarchy  describes  how  the 
system  agents  interact  with  each  other,  hiding  the  details  of  interaction  between 
sub-agents.  The  behavioral  hierarchy  describes  behavior  of  each  agent,  hiding  the 
low-level  behavioral  details.  In  our  example,  we  have  only  one  level  of  architecture 
description  with  agents  Tank  and  Controller.  There  are  two  variables  shared 
by  the  agents:  level  for  the  level  of  the  liquid,  and  infusion  for  the  infusion 
rate. 

Both  agents  are  primitive,  that  is,  without  concurrent  sub-agents.  Behavior 
of  a  primitive  agent  is  given  by  a  mode,  a  hybrid  state  machine  equipped  with 
analog  and  discrete  variables.  While  a  mode  stays  in  a  state,  its  analog  variables 
are  updated  continuously  according  to  a  set  of  constraints.  Taking  transitions 
from  one  state  to  another,  the  mode  updates  its  discrete  variables.  States  of  the 
mode  are  submodes  that  can  have  their  own  behavior.  A  mode  has  a  number 
of  control  points,  through  which  control  enters  and  exits  the  mode.  That  is,  to 
perform  a  computation  in  one  of  its  submodes,  a  mode  takes  a  transition  to  an 
entry  point  of  that  submode.  When  the  computation  is  complete,  a  transition 
from  an  exit  point  of  the  submode  is  taken.  Before  the  computation  of  a  mode 
is  completed,  it  may  be  interrupted  by  a  group  transition,  originating  from  a 
default  exit  point  dx.  After  an  interrupt,  control  is  restored  to  the  mode  via  a 
default  entry  point  de.  In  our  example,  the  behavior  of  Tank  is  represented  by 
a  single  differential  equation  d{level)  =  infusion  —  leak,  where  leak  is  a  local 
variable  of  Tank.  Figure  1  shows  the  behavior  of  the  agent  Controller.  The 
top>-level  mode  of  Controller  has  two  submodes.  Normal  and  Emergency.  We 
do  not  show  the  details  of  the  mode  Emergency.  It  is  activated  when  the  level 
enters  the  critical  region. 


Compute 

local  discrete  real  est 

global  analog  real  level,  infusion 

global  discrete  real  rate 


ComputeHigh 

diinfusion)  =  est-1 


de  A  ComputeLow 
d(  infusion)  =  est+J 


Fig.  1.  Behavior  of  the  controller 


The  mode  Normal  has  two  submodes.  Submode  Maintain  is  used  to  maintain 
the  current  rate  of  change  for  infusion,  represented  by  a  local  variable  rate.  Every 
10  seconds,  measured  by  a  local  clock  t,  Maintain  makes  a  call  to  Submode 
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Compute  that  computes  a  new  value  of  rate.  The  details  of  the  computation  are 
irrelevant,  but  we  assume  that  the  computation  is  done  differently  depending  on 
the  level  We  therefore  introduce  two  submodes  in  Compute  and  show  only  the 
constraints  for  infusion  in  each  submode.  The  exit  transition  of  Compute  assigns 
the  computed  value  to  the  variable  rate. 

Note  that  the  mode  Normal  controls  the  value  of  the  clock  t,  and  its  rate 
of  change  is  the  same  in  all  its  submodes.  By  contract,  infusion  is  updated 
differently  in  the  two  submodes.  In  this  case,  every  submode  must  provide  a 
constraint  for  infusion.  Note  also  that  rate  is  a  discrete  variable.  It  is  updated 
only  by  transitions  of  Compute. 

We  use  invariants  to  force  one  of  the  outgoing  transitions.  Control  can  reside 
in  a  mode  only  as  long  as  its  invariant  is  satisfied.  As  soon  as  an  invariant  is 
violated,  control  has  to  leave  the  mode  by  taking  one  of  the  enabled  outgoing 
transitions.  In  Figure  1,  invariants  of  the  modes  are  shown  in  braces.  For  exam¬ 
ple,  ten  time  units  after  entering  the  mode  Maintain  the  transition  to  Compute 
has  to  be  taken. 

We  distinguish  between  regular  transitions  and  interrupts.  For  example,  con¬ 
trol  is  transferred  from  Compute  to  Maintain  only  when  the  computation  is  com¬ 
plete.  When  it  is  time  to  perform  another  computation,  it  will  start  from  the 
beginning.  On  the  other  hand,  the  transition  from  Normal  to  Emergency  works 
as  an  interrupt.  Regardless  of  which  submode  of  Normal  is  operating  when  an 
interrupt  occurs,  control  is  transferred  to  Emergency.  Upon  return  from  the  in¬ 
terrupt,  the  control  state  of  Normal  is  restored.  There  is  no  priority  between 
regular  transitions  and  interrupts^.  A  mode  can  ignore  an  enabled  interrupt  and 
execute  its  internal  transitions  or  let  time  elapse.  We  use  invariants  as  described 
above  to  enforce  interrupts  (see  the  invariant  of  mode  Normal).  Invariants  give 
the  user  finer  control  over  interrupts.  For  example,  a  situation  when  an  interrupt 
is  optional  for  some  time  and  then  becomes  urgent  can  be  easily  expressed. 

In  addition  to  discrete  steps  described  above,  a  mode  can  make  continuous 
steps,  when  time  progresses  and  the  analog  variables  of  the  mode  are  updated 
according  to  a  set  of  constraints.  Because  of  the  hierarchical  structure  of  the 
mode,  the  set  of  applicable  constraints  consists  of  the  constraints  defined  in  the 
mode  itself  and  those  from  the  currently  active  submode.  This  implies  that  a 
mode  can  engage  in  a  continuous  step  only  when  its  control  properly  resides 
within  one  of  its  submodes.  For  example,  we  cannot  allow  time  to  pass  at  the 
control  point  e  of  Compute,  between  executing  the  transition  from  Maintain  to 
Compute  and  a  transition  to  enter  ComputeHigh  or  ComputeLow. 


3  Modes 

Notation.  We  will  represent  modes  and  agents  as  tuples  of  components.  If  T  is 
a  tuple  (ti, . . .  ,tn),  we  identify  the  component  U  of  T  as  T.U.  We  extend  this 


^  Other  treatments  of  interrupts  can  be  handled  equally  well  within  the  proposed 
framework.  For  example,  [2]  discuss  weak  interrupts  in  a  similar  setting. 
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notation  to  sets  of  tuples.  If  ST  is  a  set  of  tuples  with  the  same  structure,  we 
write  ST.ti  to  mean 

Given  a  set  V  of  typed  variables,  a  valuation  for  K  is  a  function  mapping 
variables  to  their  values.  We  will  assume  that  all  valuations  are  type  correct. 
The  set  of  valuations  over  V  is  denoted  Qy.  We  will  use  variables  s,t,  possibly 
primed  or  subscripted,  to  range  over  valuations.  Given  a  valuation  s  over  V,  and 
a  set  W  C  K,  s[W]  denotes  the  restriction  of  s  to  the  variables  of  W. 

A  flow  for  a  set  V  of  variables  is  a  differentiable  function  /  from  a  closed 
interval  of  non-negative  reals  [0,<^]  to  Qy.  We  refer  to  S  as  the  duration  of  the 
flow.  We  assume  that  only  constant  functions  are  differentiable  for  non  real¬ 
valued  types.  We  denote  a  set  of  flows  for  V  as  Ty. 

3.1  Synt£Lx 

Definition  1.  (Mode)  A  mode  M  is  a  tuple  (E,X,V,SM,  Cons,T),  where  E 
is  a  set  of  entry  control  points,  X  is  a  set  of  exit  control  points,  V  is  a  set  of 
variables,  SM  is  a  set  of  submodes,  Cons  is  a  set  of  constraints,  and  T  is  a  set 
of  transitions. 

Variables.  A  mode  has  a  finite  set  of  typed  variables  V,  partitioned  into  subsets 
Va  and  Vd,  the  sets  of  analog  and  discrete  variables,  respectively.  We  also  parition 
V  into  Vg  and  VJ,  the  sets  of  global  and  local  variables^.  We  assume  that  there 
are  no  conflicts  between  the  names  of  local  variables  of  different  modes. 
Submodes.  SM  is  a  finite  set  of  submodes.  We  require  that  each  global  variable 
of  a  submode  is  a  variable  (either  global  or  local)  of  its  parent  mode.  That  is,  if 
N  ^  SM,  then  N.Vg  CV.  This  induces  a  natural  scoping  rule  for  variables  in  a 
hierarchy  of  modes:  a  variable  introduced  as  local  in  a  mode  is  accessible  in  all 
its  submodes  but  not  in  any  other  mode. 

Control  points.  E  is  the  set  of  entry  points',  X  is  the  set  of  exit  points.  There 
are  two  distinguished  control  points  representing  default  entry  and  exit;  de  G  E 
and  dx  ^  X.  We  use  C  for  the  set  of  all  control  points  of  the  mode:  C  = 
EUXU  SM.E  U  SM.X. 

Constraints.  The  finite  set  Cons  of  constraints  defines  the  flows  permitted  by 
M^.  Cons  contains  an  invariant  I,  which  defines  when  the  mode  can  be  active 
(see  the  definition  of  an  active  mode  below).  Further,  for  a  variable  x  ^  Va, 
Cons  can  contain  an  algebraic  constraint  A^,  which  defines  the  set  of  admissible 
values  for  x,  or  a  differential  constraint  which  defines  admissible  values 
for  the  derivative  of  x  with  respect  to  time.  Every  invariant  and  an  algebraic 
constraint  is  a  predicate  cC  Qy  and  a  differential  constraint  Dx  is  a  predicate 
on  Qyud{V)‘  A  flow  /  is  permitted  by  the  mode  if  for  every  t  in  the  domain  of  /, 
every  variable  in  f(t)  satisfies  all  constraints  in  Cons.  Examples  of  constraints 
are  d{x)  <  f{x,  y)  and  g{x,  y)  <  0. 

Charon  refines  the  set  of  global  variables  further  according  to  allowed  read/write 
access,  but  we  won’t  make  such  a  distinction  in  this  paper  for  clarity  of  presentation. 

^  The  semantics  does  not  depend  on  how  sets  of  flows  are  specified.  Here,  we  chose 
one  of  the  possible  ways. 
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Transitions.  T  is  a  finite  set  of  transitions  of  the  form  {e,a,x),  where  e  e 
E  U  SM.X,  X  e  X  U  SM,E,  and  o,  the  action  of  the  transition,  is  a  relation 
from  Qvg  to  Qv  ^  e  E  and  from  Qv  to  Qv  otherwise.  A  transition  connects 
control  points  of  the  mode  or  its  submodes.  When  a  transition  is  executed,  it 
updates  some  variables  of  the  mode.  Every  mode  is  assumed  to  have  an  identity 
transition  from  de  to  dx,  but  we  disallow  transitions  from  any  non- default  control 
point  to  dx.  A  transition  that  originates  at  a  default  exit  point  of  a  submode  is 
called  a  group  transition  of  that  submode.  A  group  transition  can  be  executed 
to  interrupt  the  execution  of  the  submode.  We  require  that  if  a  submode  has 
been  exited  by  a  group  transition,  it  must  be  entered  again  through  its  default 
entry  point  to  resume  the  interrupted  execution. 

Furthermore,  we  require  that  the  mode  cannot  be  blocked  at  any  of  its  non¬ 
default  control  points.  Precisely,  for  every  eot  M  that  is  not  de  in  M  or  dx  in  one 
of  the  submodes  of  M,  the  union  ttg  of  all  actions  of  the  transitions  originating 
at  e  is  complete,  that  is,  for  every  s  there  is  t  such  that  {s,t)  G  ae- 
Special  modes.  We  distinguish  two  kinds  of  modes  that  play  a  special  role  in 
the  semantic  definitions.  A  mode  M  is  a  leaf  mode  if  M.5M  =  0.  Leaf  modes 
perform  continuous  steps  according  to  their  constraints.  A  top-level  mode  has 
a  single  non-default  entry  point  init  and  no  non-default  exit  points.  Top-level 
modes  are  used  to  describe  behavior  of  agents,  as  shown  in  Section  4. 


3.2  Semantics 

Intuition.  A  mode  can  engage  in  a  discrete  or  continuous  behavior.  During 
an  execution,  the  mode  and  its  environment  either  take  turns  making  discrete 
steps  or  take  a  continuous  step  together.  Discrete  and  continuous  steps  of  the 
mode  alternate.  During  a  continuous  step,  the  mode  follows  a  flow  from  the  set 
of  flows  possible  for  the  current  state  for  the  length  of  its  duration,  updating 
its  variables  according  to  the  flow.  Note  that  the  set  of  flows  permitted  by  the 
mode’s  constraints  may  be  further  restricted  by  the  mode’s  environment.  A 
discrete  step  of  the  mode  is  a  finite  sequence  of  discrete  steps  of  the  submodes 
and  enabled  transitions  of  the  mode  itself.  A  discrete  step  begins  in  the  current 
state  of  the  mode  and  ends  when  it  reaches  an  exit  point  or  when  the  mode 
decides  to  yield  control  to  the  environment  and  let  it  make  the  choice  of  the 
next  step.  Note  that  in  the  latter  case,  the  decision  to  break  a  discrete  step  is 
made  by  the  mode  itself.  Technically,  when  the  mode  ends  its  discrete  step  in 
one  of  its  submodes,  it  returns  control  to  the  environment  via  its  default  exit 
point.  The  closure  construction,  described  below,  ensures  that  the  mode  can 
yield  control  at  appropriate  moments,  and  that  the  discrete  control  state  of  the 
mode  is  restored  when  the  environment  schedules  the  next  discrete  step. 

State  of  a  mode.  We  define  the  state  of  a  mode  in  terms  of  all  variables  of  the 
mode  and  its  submodes.  We  use  =  V  U  SM.V^  for  the  set  of  all  variables. 

The  state  of  a  mode  M  is  a  pair  (c,  s),  where  c  is  the  location  of  discrete 
control  in  the  mode  and  s  G  Qm.v*-  Whenever  the  mode  has  control,  it  resides 
in  one  of  its  control  points.  In  this  case,  c  G  M.C.  We  use  special  symbol  e  to 
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denote  the  case  when  the  mode  does  not  have  control  Given  a  state  (c,  s)  of  M, 
we  refer  to  c  as  the  control  state  of  M  and  to  s  as  the  data  state  of  M. 
Preemption.  An  execution  of  a  mode  can  be  preempted  by  a  group  transition. 
A  group  transition  of  a  mode  originates  at  the  default  exit  of  the  mode.  During 
any  discrete  step  of  the  mode,  control  can  be  transferred  to  the  default  exit  and 
an  enabled  group  transition  can  be  selected.  There  is  no  priority  between  the 
transitions  of  a  mode  and  its  group  transitions.  When  an  execution  of  a  mode  is 
preempted,  the  control  state  of  the  mode  is  recorded  in  a  special  history  vaxiahle, 
a  new  local  variable  that  we  introduce  into  every  mode.  Then,  when  the  mode  is 
entered  through  the  default  entry  point  next  time,  the  control  state  of  the  mode 
is  restored  according  to  the  history  variable. 

The  history  variable  and  active  submodes.  In  order  to  record  the  location 
of  discrete  control  during  executions,  we  introduce  a  new  local  variable  h  into 
each  mode  that  has  submodes.  The  history  variable  h  of  sl  mode  Af  can  assume 
values  from  the  set  SMUe.  A  submode  AT  of  M  is  called  active  when  the  history 
variable  of  M  has  the  value  N,  Every  top-level  mode  is  always  active. 

Closure  of  a  mode.  Closure  construction  is  a  technical  device  to  allow  the  mode 
to  interrupt  its  execution,  either  to  allow  the  environment  to  schedule  another 
step  or  to  provide  for  preemption  of  the  mode  execution  by  group  transitions. 
Transitions  of  the  mode  are  modified  to  update  h  after  a  transition  is  executed. 
In  addition,  default  entry  and  exit  transitions  are  added  to  the  set  of  transi¬ 
tions  of  the  mode.  These  default  transitions  do  not  affect  the  history  variable 
and  allow  us  to  interrupt  an  execution  and  then  resume  it  later  from  the  same 
point. 

The  closure  modifies  the  transitions  of  M  in  such  a  way  that,  after  each 
transition,  h  records  the  active  submode.  If  a  transition  leads  to  a  control  point 
of  a  submode  N,  the  resulting  state  has  h  =  N.  Otherwise,  if  the  transition 
leads  to  a  control  point  of  M  itself,  the  value  of  h  after  the  transition  will  be 
e.  For  each  submode  N  of  M,  the  closure  adds  a  default  exit  transition  from 
N.dx  to  M.dx.  This  transition  does  not  change  any  variables  of  the  mode  and 
is  always  enabled.  Default  entry  transitions  are  used  to  restore  the  local  control 
state  of  M.  A  default  entry  transition  leads  from  a  default  entry  of  the  mode  to 
the  default  entry  of  every  submode  N  and  is  enabled  iih  =  N.  Furthermore,  we 
make  sure  that  the  default  entry  transitions  do  not  interfere  with  regular  entry 
transitions  originating  from  de.  The  closure  changes  each  such  transition  so  that 
it  is  enabled  only  ii  h  =  e. 

Formally,  the  closure  c(M)  of  a  mode  M  =  (E,  X,  F,  SM,  Cons,  T)  is  defined 
to  be  the  mode  (E,  X,  V  U/i,  c{SM),  Cons,  c{T)),  where  h  is  a  new  local  vari¬ 

able,  c{SM)  —  {c{m)  I  m  G  SM}  is  the  set  of  closed  submodes  of  M,  and  c(T)  is 
the  closed  set  of  transitions  obtained  by  extending  T  with  transitions  {x,  dx) 
for  every  x  e  SM.dx  and  {de,aj:,e)  for  every  e  G  SM.de,  and  extending  every 
transition  in  T  such  that 

-  (5,  s)  G  Oa;  iff  X  G  N.E  for  some  N  G  SM  and  s[/i]  =  N; 

~  for  every  transition  (e,  a,  x)  G  T,  the  respective  closed  transition  is  (e,  a',  x), 

where  {s,t)  G  a'  iff  {s[V],t[V])  G  a  and 
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-  \ix  e  N.E  for  some  N  €  SM,  then  t[h]  =  N,  otherwise  t[h]  =  e, 

-  if  e  e  N.X  for  some  N  e  SM,  then  s[h]=  N,  otherwise  s[h]  —  e. 

The  closure  construction  for  the  example  introduced  in  Section  2  is  illustrated 
in  Figure  2.  To  avoid  cluttering  the  figure,  we  omit  the  default  transitions  of  the 
submode  ComputeLow,  and  do  not  show  the  variables  of  the  modes. 


I  dx 


Compute 
h=ComputeHigh 

ComputeHigh 
d(  infusion)  =  est-1 

h-*-  ComputeHigh 
est  -*■ ...  fate  ■ 

level  >  5  A  A  =  £  h- 

level  <  5  A.  h-t  h- 


h-*-  ComputeLow 
ComputeLow 
d(  infusion)  =  est+1 


Fig.  2.  Closed  modes 


Before  formally  defining  executions  of  a  mode,  we  illustrate  continuous  and 
discrete  steps  using  the  example  in  Figure  2.  Assume  that  the  the  controller 
is  in  the  Maintain  mode  and  none  of  the  invariants  is  violated.  Maintain  can 
voluntarily  relinquish  control  to  the  environment  to  let  it  take  a  step  or  advance 
time  by  taking  the  default  exit  transition  to  dx  of  Normal.  There,  the  group 
transition  is  not  enabled,  and  the  default  exit  transition  of  the  parent  mode 
is  taken.  When  the  control  arrives  thus  at  the  top  level,  the  environment  can 
schedule  a  continuous  step.  The  analog  variables  of  all  agents  are  updated  ac¬ 
cording  to  the  constraints  of  the  active  modes.  The  active  modes  are  Maintain, 
Normal,  and  Controller.  Thus,  the  applicable  constraints  are  d{t)  —  1  and 
d{in fusion)  =  rate.  The  global  variable  level  is  updated  according  to  the  con¬ 
straint  in  Tank.  After  the  continuous  step,  control  returns  to  Maintain  via  the 
chain  of  default  entry  transitions.  Assume  now  that  the  invariant  of  Normal  is  vi¬ 
olated  while  control  is  inside  a  submode  of  Compute.  Then,  control  is  transferred 
to  dx  of  Compute  and  then  on  to  dx  of  Normal.  There,  the  choice  between  the 
group  transition  to  Emergency  or  the  default  exit  transition  is  non-deterministic. 
But  since  the  invariant  is  violated,  a  continuous  step  cannot  be  taken. 
Operational  semantics.  An  operational  view  of  a  closed  mode  M  with  the  set 
of  variables  V  consists  of  a  continuous  relation  and,  for  each  pair  ci  £  E, 
C2  G  X,  a  discrete  relation  -R^,C2* 

The  relation  CQyxEv  gives,  for  every  data  state  of  the  mode,  the  set  of 
flows  from  this  state.  By  definition,  if  the  control  state  of  the  mode  is  not  at  dx, 
the  set  of  flows  for  the  state  is  empty.  We  require  that,  whenever  (s,  /)  G  R^ , 
/(O)  —  s.  In  addition,  for  each  s,  the  set  of  flows  Es  =  {f  \  (-s,/)  ^  R^}  is 
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prefix-closed.  That  is,  if  the  domain  of  f  ^  Tg  is  [0,  J],  then  for  every  e  <  6,  a 
flow  /'  :  [0,6]  that  coincides  with  /  on  [0,e]  also  belongs  to  is  obtained 

from  the  constraints  of  a  mode  and  relations  SM.R^  of  its  submodes.  Given  a 
data  state  s  of  a  mode  M,  (s,  /)  e  RP  iff  f  is  permitted  by  M  and,  if  N  is  the 
active  submode  at  s,  (s[Ar.y], /[AT.y])  ^  N.Rp . 

For  each  ci  e  E  \J  SM.X,  e  X  U  SM.E,  relation  C  Qy  x  Qv 

describes  the  discrete  behavior  in  which  control  is  transferred  from  ci  to  C2. 
The  relation  R^^  comprises  macro-steps  of  a  mode  starting  at  e  and  ending 
at  a:.  A  macro  step  consists  of  a  sequence  of  micro-steps.  Each  micro-step  is 
either  a  transition  of  the  mode  or  a  macro-step  of  one  of  its  submodes.  Given 
the  relations  e'  G  SM.E,  x'  G  SM.X  of  macro-steps  of  the  submodes  of 

M,  a  micro-execution  of  a  mode  M  =  {E,  X,V,  SM,C,T)  is  a  sequence  of  the 
form  (eo,so),(ei,si),...  ,(e^,Sn)  such  that,  for  all  i,  Ci  e  C  and  G  K  and 
for  even  i,  ((e^, s^),  (e^+i, s^+i))  G  T,  while  for  odd  i,  {si,Si+i)  G 
Given  such  a  micro  execution  of  M  with  eo  =  e  G  E  and  e„  =  a:  G  X,  we ’have 
{so^Sn)  G  R^^^. 

Definition  2.  (Operational  semantics)  The  operational  semantics  of  the  mode 
M  consists  of  its  control  points  EU  X,  its  variables  V  and  relations  R^  and 

The  operational  semantics  of  a  mode  defines  a  transition  system  R  over 
the  states  of  the  mode.  We  write  (ei,  Si)-^(e2,  S2)  if  (si,S2)  G  ^  ,  and 

/  1  ’  2 

{dx,  si)^{dx,  S2)  if  (si,  /)  G  R^ ,  f  is  defined  on  the  interval  [0,  t]  and  f{t)  =  S2. 
We  extend  R  to  include  environment  steps.  An  environment  step  begins  at  an 
exit  point  of  the  mode  and  ends  at  an  entry  point.  It  represents  changes  to  the 
global  variables  of  the  mode  by  other  components  while  the  mode  is  inactive. 
Private  variables  of  the  mode  are  unaffected  by  environment  steps.  Thus  there 
is  an  environment  step  (a:,s)4(e,t)  whenever  rr  G  X,  e  G  E,  and  s[Vp]  =  t[Vp]. 
We  let  A  range  over  Ev  U  {o,  e}.  An  execution  of  a  mode  is  now  a  path  through 
the  graph  of  R: 


(^0,  5o)4(ei,  si)4  . . .  4(6^1,  s^). 


3.3  Trace  Semantics 

To  be  able  to  define  a  refinement  relation  between  modes,  we  consider  a  trace 
semantics  for  modes.  A  trace  of  the  mode  is  a  projection  of  its  execution  onto 
the  global  variables  of  the  mode.  That  is,  a  trace  is  obtained  from  each  execution 
by  replacing  every  Si  with  every  /  in  transition  labels  with  /[V^].  We 

denote  the  set  of  traces  of  a  mode  M  by  Lm- 

Definition  3.  (Trace  semantics  for  modes)  The  trace  semantics  for  M  is  given 
by  its  control  points  E  and  X,  its  global  variables  V,  and  its  set  of  its  traces  Lm- 

In  defining  compositional  and  hierarchical  semantics,  one  has  to  decide,  what 
details  of  the  behavior  of  lower-level  components  are  observable  at  higher  levels. 
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In  our  approach,  the  effect  of  a  descrete  step  that  updates  only  local  variables 
of  a  mode  is  not  observable  by  its  environment,  but  stoppage  of  time  introduced 
by  such  step  is  observable.  For  example,  consider  two  systems,  one  of  which  is 
always  idle,  while  the  other  updates  a  local  variable  every  second.  These  two 
systems  are  different,  since  the  second  one  does  not  have  flows  more  than  one 
second  long.  Defining  a  modular  semantics  in  a  way  that  such  distinction  is  not 
made  seems  much  more  difficult. 


4  Agents 

4.1  Syntax 

Definition  4.  (Agent)  An  agent  {TM,V^I)  consists  of  a  set  of  variables  V,  a 
set  of  initial  states,  and  a  set  of  top-level  modes  TM. 

The  top-level  modes  collectively  define  behavior  of  the  agent.  The  set  V 
is  partitioned  into  local  variables  Vj  and  global  variables  Vg.  We  require  that 
TM.V  ^V,VgC  TMVg]  that  is,  all  global  variables  originate  in  some  mode. 
The  set  of  initial  states  I  C  Qv  specifies  possible  initializations  of  the  variables 
of  the  agent.  A  primitive  agent  has  a  single  top-level  mode.  Composite  agents 
have  many  top-level  modes  and  are  constructed  by  parallel  composition  of  other 
agents  as  described  below. 

4.2  Semantics 

An  execution  of  an  agent  follows  a  trajectory,  which  starts  in  one  of  the  initial 
states  and  is  a  sequence  of  flows  interleaved  with  discrete  updates  to  the  variables 
of  the  agent.  An  execution  of  A  is  constructed  from  the  relations  and  of 
its  top-level  modes.  For  a  fixed  initial  state  so,  each  mode  M  G  TM  starts  out 
in  the  state  where  initM  is  the  non-default  entry  point  of  M  and 

so[M.V]  =  sM’  Note  that  as  long  as  there  is  a  mode  M  whose  control  state  is  at 
initM,  no  continuous  steps  are  possible.  However,  any  discrete  step  of  such  mode 
will  come  from  R^nitMAx  bring  the  control  state  of  M  to  dx.  Therefore,  any 
execution  of  an  agent  A  =  {TM,V,I)  with  \TM\  =  k  will  start  with  exactly 
k  discrete  initialization  steps.  At  that  point,  every  top-level  mode  of  A  will  be 
at  its  default  exit  point,  allowing  an  alternation  of  continuous  steps  from  Rp 
and  discrete  steps  from  R^e  dx’  choice  of  a  continuous  step  involving  all 
modes  or  a  discrete  step  in  one  of  the  modes  is  left  to  the  environment.  Before 
each  discrete  step,  there  is  an  environment  step,  which  takes  the  control  point 
of  the  chosen  mode  from  dx  to  de  and  leaves  all  the  private  variables  of  all 
top-level  modes  intact.  After  that,  a  discrete  step  of  the  chosen  mode  happens, 
bringing  control  back  to  dx.  Thus,  an  execution  of  A  with  \TM\  =  fc  is  a  sequence 

So — ^ . . .  s/^ — ^  . . .  such  that 

”  for  every  0  <  f  <  A:,  there  is  M  e  TM  such  that  [silM.V],  Si+i[M.V])  G 
M.R^^tM,dx’  That  is,  the  first  k  steps  initialize  the  top-level  modes  of  A. 
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-  for  every  i  >  k,  one  of  the  following  holds: 

-  SiA-Si^i  such  that  /  is  defined  on  [0,t]  and  f{t)  =  Si+i,  and  for  ev¬ 
ery  mode  M  e  TM,  {si[M.V]J[My])  e  M,R^\  that  is,  the  step  is  a 
continuous  step,  in  which  every  mode  takes  part; 

-  SiAsi+i  such  that  for  every  mode  M  e  TM,  Si[M.Vp]  =  Si+i[M.V^]; 
that  is,  the  step  is  an  environment  step; 

-  with  i>  there  is  M  e  TM  such  that  {si{My],  Si+i[M.V])  e 

^'^de,dx'^  step  is  a  discrete  step  by  one  of  the  modes. 

Note  that  environment  steps  in  agents  and  in  modes  are  different.  In  an  agent, 
an  environment  step  may  contain  only  discrete  steps,  since  all  agents  participate 
in  every  continuous  step.  The  environment  of  a  mode  can  engage  in  a  number 
of  continuous  steps  while  the  mode  is  inactive. 

Definition  5.  (Trace  semantics  for  agents)  A  trace  of  A  is  an  execution  of  A, 
projected  onto  the  set  of  its  global  variables.  The  denotational  semantics  of  an 
agent  consists  of  its  set  of  global  variables  and  its  set  of  traces. 

Let  A  be  a  primitive  agent  and  {init,  so)-^(dx,  si)^(c2,  52)^  •  •  •  Sn) 

be  a  trace  of  its  top-level  mode.  It  is  easy  to  see  that  5oAsi^S2^  . . .  is  a 

trace  of  A.  A  similar  statement  is  true  for  agents  with  multiple  top-level  modes. 

4.3  Operations  on  Agents 

Variable  hiding.  The  hiding  operator  makes  a  set  of  agent  variables  private. 
Given  an  agent  A  =  {TM,V,I),  the  agent  A\{14}  =  (TM.V'J)  with  V/  = 
=  Vg  —  Vh’  A  trace  of  A,  projected  onto  the  set  of  global  variables  of 
A\{V/i},  is  a  trace  of  A\{V/i}. 

Variable  renaming.  Variable  renaming  replaces  a  set  of  variables  in  an  agent 
A  with  another  set  of  variables.  Let  Vi  =  {xi,...  ,Xn},V2  =  {2/1,...  ,2/n}  be 
indexed  sets  of  variables  with  Vi  C  A.V.  Then,  A[Vi  :=  V2]  is  an  agent  with 
the  set  of  global  variables  {A.Vg  —  Vi)  U  V2.  Semantics  of  the  variable  renaming 
operator  is  given  by  renaming  the  variables  in  the  traces  of  the  agent. 

Parallel  composition.  The  composition  of  the  two  agents  Ai|jA2  is  an  agent 
A  =  {TM,V,I)  defined  as  follows:A.TM  =  Ai.TM  U  A2.TM,  A.V^  =  Ai.V^  U 
A2-V^,  A.Vi  =  Ai.Vi  U  A2.Vi,  and  if  s  e  A.I  then  sfAi.V]  G  Ai.I  and  s[A2.V]  G 
A2.I. 

5  Compositionality  Results 

We  show  that  our  semantics  is  compositional  for  both  modes  and  agents.  First, 
the  set  of  traces  of  a  mode  can  be  computed  from  the  definition  of  the  mode 
itself  and  the  semantics  of  its  submodes.  Second,  the  set  of  traces  of  a  composite 
agent  can  be  computed  from  the  semantics  of  its  sub-agents.  For  the  lack  of 
space,  we  omit  the  proofs  and  concentrate  on  intuitions  for  the  results. 
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g<(g)  ...  g<(g) 


< 

M 

M 

c, 

0-0. 

C, 

Fig.  3.  Compositionality  rules  for  modes 


5.1  Compositionality  of  Modes 

In  order  to  show  that  our  trace  semantics  for  modes  is  compositional,  we  need 
to  be  able  to  define  the  semantics  of  a  mode  only  in  terms  of  the  semantics  of 
its  submodes. 

Compositional  Trace  Construction.  First,  we  show  that  every  trace  of  a 
mode  can  be  constructed  using  the  traces  of  the  submodes. 

Theorem  1.  The  set  of  traces  of  a  mode  M  can  be  computed  from  the  set  of 
traces  of  its  submodes,  its  closed  transition  relation  c(T)  and  the  set  of  con¬ 
straints  Cons. 

Theorem  1  relies  on  the  following  observation.  Given  a  submode  AT  of  M,  we 
can  “project”  a  trace  cr  of  M  onto  N  and  obtain  a  trace  of  iV.  This  projection 
will  1)  restrict  all  data  states  and  flows  to  the  global  variables  of  iV,  2)  replace 
every  subsequence  of  a  where  N  is  inactive  into  a  single  environment  step,  and  3) 
convert  continuous  steps  of  M  into  continuous  steps  of  N  by  removing  transitions 
from  N.dx  to  M.dx  and  from  M.de  to  N.de.  The  critical  point  in  proving  this 
observation  is  that,  whenever  the  control  state  is  at  dx  of  M,  and  N  is  the 
active  submode  of  M,  N  has  its  control  state  at  N.dx,  since  only  default  exit 
transitions  and  the  identity  transition  of  the  mode  can  end  at  dx. 

Mode  Refinement.  The  trace  semantics  leads  to  a  natural  notion  of  refinement 
between  modes:  a  mode  M  refines  N  if  it  has  the  same  global  variables  and 
control  points,  and  every  trace  of  M  is  a  trace  of  N. 

Definition  6.  (Refinement)  A  mode  M  and  a  mode  N  are  said  to  be  compatible 
if  M.Vg  —  N.Vg,  M.E—N.E  and  M.X=N.X .  Given  two  compatible  modes  M 
and  N,  M  refines  N,  denoted  M^N ,  if 

For  a  finite  index  set  /,  we  write  {Mi  |  i  6  /}  d  {Ni  \  i  ^  I]  if  Mi  <  Ni 
for  each  i  E  I.  The  refinement  operator  is  compositional  with  respect  to  the 
encapsulation: 

Theorem  2.  (Submode  compositionality)  Given  a  mode  N,  suppose  SM  ■<  SN 
and  let  M  =  N[SM/SN].  Then  M  ^  N. 

The  refinement  rule  is  explained  visually  in  Figure  3,  left.  If  we  consider  a  sub¬ 
mode  N  within  a  mode  M,  the  remaining  submodes  of  M  and  the  transitions 
of  M  can  be  viewed  as  an  environment  or  mode  context  for  N.  In  other  words, 
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a  context  for  A^i . . ,  Nk  is  a  mode  M[Gi, . . .  G^]  with  holes  or  most  general  sub¬ 
modes  Gj,  l<z<fc  that  have  the  same  interface  as  have  no  local  variables  and 
put  no  constraints  on  the  update  of  global  variables.  Two  contexts  are  said  to 
be  compatible  if  they  are  compatible  as  modes  and  they  also  are  compatible  on 
their  holes. 

Definition  7.  (Context  traces)  An  execution  of  a  mode  context  C  with  holes 
Gi . . .  Gk  is  a  path 

(eo,  so)^{ei,$i)^  . . .  h{en,  Sn) 

through  the  graph  of  11  of  C  with  \i  =  e  for  each  e^,  such  that  is  in  C.X 
and  Ci-i-i  is  in  C.E  or  Ci  is  in  Gj.E  and  Ci^i  is  in  Gj.X ,  for  l<j<k.  A  trace 
of  C  is  obtained  by  projecting  an  execution  on  its  global  variables. 

As  with  modes,  the  set  of  traces  of  a  context  C  is  denoted  by  Lc  and  refinement 
is  defined  by  language  inclusion.  Given  a  context  C  with  holes  Gi,...Gk  and 
a  set  of  modes  Ni,...Nk  such  that  Ni  ^  Gi  for  l<i<k,  we  write  C[Ni, . . .  Nk] 
the  mode  obtained  by  filling  the  holes  Gi  of  C  with  Ni.  Contexts  are  also  com¬ 
positional. 

Theorem  3.  (Context  compositionality)  Let  Ci  and  C2  be  compatible  contexts 
with  holes  Gi  ...Gk.  If  Ci  ^  C2  then  Ci[Ni, ...  ,Nk]:<  C2[Ni, ...  ,Nk]  for  any 
set  Ni,  l<i<k  of  modes  compatible  with  the  holes,  i.e.,  Ni  ^  Gi  for  all  i. 

A  visual  representation  of  this  rule  is  shown  in  Figure  3,  right.  The  compo¬ 
sitionality  rules  allow  us  to  decompose  the  proof  obligation  into  refinement  of 
submodes  in  the  most  general  context,  and  refinement  of  contexts  under  the 
most  general  submode. 


Fig.  4.  Refinement  example 


Consider  mode  Normal  in  Figure  1  as  a  two-place  context.  Let  Normal^ 
differ  from  Normal  only  by  allowing  rate  computation  to  happen  more  often. 
The  transition  to  Compute  has  a  relaxed  guard  i  <  10,  as  shown  in  Figure  4. 
By  Theorem  3,  Normal[Maintain, Compute]  ^  Normal  ’  [Maintain, Compute].  If 
Controller'  is  the  agent  in  which  Normal'  replaces  Normal,  then  by  Theorem  2, 
Controller  ^  Controller'. 
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5.2  Compositionality  of  Agents 

An  agent  is,  in  essence,  a  set  of  top  level  modes  that  interleave  their  discrete 
transitions  and  synchronize  their  flows,  the  compositionality  results  for  modes 
lift  in  a  natural  way  to  agents  too.  The  operations  on  agents  are  compositional 
with  respect  to  refinement. 

Definition  8.  (Refinement)  An  agent  A  and  an  agent  B  are  said  to  be  com¬ 
patible  if  A.Vg  =  B.Vg.  Agent  A  refines  a  compatible  agent  B,  denoted  A-<B,  if 


Theorem  4.  (Agent  compositionality)  Given  compatible  agents  such  that 
A^B.AidiBi  and  A2^B2.  Let  Vi  =  {xi,...  ,Xn},V2  =  -  ,2/n}  be  in¬ 

dexed  sets  of  variables  with  Vi  C  A.V  and  let  Vh  C  A.V.  Then  A\{14} 
B\{Vh},A[Vi  :=  V2]  ^  B[V,  14]  and  Ai||A2  Bi\\B2 

In  our  example,  Tank|| Controller  ■<  Tajik||Controller  ^  by  Theorem  4. 

6  Conclusions 

We  have  presented  a  hierarchical  modular  semantics  for  hybrid  systems.  The 
proposed  semantics  is  compositional  both  with  respect  to  the  system  architec¬ 
ture  (parallel  agents  and  their  subagents)  and  the  system  behavior  (modes  and 
their  submodes).  We  have  introduced  the  notion  of  refinement  between  the  sys¬ 
tem  components  -  both  modes  and  agents  -  and  showed  that,  in  the  proposed 
semantics,  composition  of  components  preserves  refinement. 

We  are  currently  working  to  build  upon  the  presented  compositionality  re¬ 
sults  and  provide  assume-guarantee  proof  rules  for  hybrid  systems,  extending 
the  results  of  [2].  The  proposed  semantics  have  been  used  in  the  modeling  lan¬ 
guage  Charon  [3]  and  its  toolkit,  currently  under  development  by  the  authors. 
For  further  details,  see 

http : //www . cis . upenn . edu/mobies/charon/. 
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Abstract.  We  consider  an  optimal-reachability  problem  for  a  timed  au¬ 
tomaton  with  respect  to  a  linear  cost  function  which  results  in  a  weighted 
timed  automaton.  Our  solution  to  this  optimization  problem  consists  of 
reducing  it  to  a  (parametric)  shortest-path  problem  for  a  finite  directed 
graph.  The  directed  graph  we  construct  is  a  refinement  of  the  region  au¬ 
tomaton  due  to  Alur  and  Dill,  We  present  an  exponential  time  algorithm 
to  solve  the  shortest-path  problem  for  weighted  timed  automata  starting 
from  a  single  state,  and  a  doubly-exponential  time  algorithm  to  solve 
this  problem  starting  from  a  zone  of  the  state  space. 


1  Introduction 

Timed  automata  [AD94]  are  widely  accepted  as  a  formalism  to  model  the  be¬ 
haviour  of  real-time  systems:  a  discrete  transition  graph  is  equipped  with  a  finite 
set  of  clock  variables  which  are  used  to  express  timing  constraints.  Automated 
analysis  of  timed  automata  relies  on  the  construction  of  a  finite  quotient  of  the 
infinite  space  of  clock  valuations.  In  particular,  this  construction  is  suitable  to 
perform  reachability  analysis.  Given  two  states  s  and  t  of  a  timed  automaton  A, 
the  reachability  problem  can  be  stated  as  the  problem  of  determining  if  there 
exists  a  run  of  A  from  s  tot.  Reachability  is  a  core  problem  in  system  verification 
and  directly  applies  to  the  verification  of  safety  properties. 

In  the  theory  of  timed  automata  there  are  many  decision  problems  which  are 
undecidable,  and  decidability  is  in  general  hard.  In  this  paper  we  are  interested  in 
an  optimal-reachability  problem  for  timed  automata.  Time-optimal  reachability 
was  first  considered  in  [CY91],  where  the  problem  of  computing  lower  and  upper 
bounds  on  time  delays  in  timed  automata  was  solved.  Minimal-time  reachability 
is  also  considered  in  [NTYOO].  In  [ACH93],  a  weight  w  is  associated  with  each 
location  q  such  that  w  gives  the  cost  of  a  unit  of  time  spent  in  q.  Then,  given 
a  cost  interval  I  and  two  states  s  and  t,  the  decision  problem  “is  t  reachable 
from  s  at  a  cost  c  G  /?”  {duration-bounded  reachability)  is  addressed  and  solved. 

*  This  work  is  partially  supported  by  the  DARPA/ITO  MoBIES  grant  F33615-00-C- 
1707,  the  NSF  Career  award  CCR97-34115,  the  SRC  award  99-TJ-688,  the  MURST 
grant  TOSCA,  the  DARPA  JFACC  grant  N66001-99-C-8510,  and  the  University  of 
Pennsylvania  Research  Foundation. 


M.D.  Di  Benedetto,  A.  Sangiovanni-Vincentelli  (Eds.);  HSCC  2001,  LNCS  2034,  pp.  49-62,  2001. 
(c)  Springer- Verlag  Berlin  Heidelberg  2001 
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Here  we  solve  a  more  general  optimal-reachability  problem,  that  has  been  inde¬ 
pendently  solved  also  in  [BHF+].  We  consider  weighted  timed  automata^  that  is 
timed  automata  with  weights  (different  costs)  on  both  locations  and  transitions. 
The  cost  of  a  run  is  given  by  the  sum  of  costs  of  the  taken  switches  plus  the  sum 
of  the  costs  associated  with  the  visited  locations  multiplied  for  the  time  spent  in 
each  of  them.  Our  optimization  problem,  which  we  call  optimal-run  problem,  can 
be  formalized  as  a  tuple  containing  a  weighted  timed  automaton,  a  source  zone 
and  a  target  zone.  If  the  source  zone  contains  only  a  state  of  the  automaton,  we 
refer  to  this  problem  as  the  single-source  optimal-run  problem. 

Our  solution  to  the  optimal-run  problem  consists  of  two  main  steps:  first  we 
reduce  the  optimal-run  problem  to  a  short est-path  problem  in  directed  graphs, 
then  we  solve  the  latter.  The  first  step  is  obtained  by  constructing  a  finite  graph 
which  is  a  refinement  of  the  region  automaton  [AD94] .  Each  clock  region  is  split 
into  several  disjoint  subregions  relatively  to  a  starting  state  and  to  sequences 
of  resets  that  may  occur  in  “potential”  optimal  runs.  This  construction  is  pa¬ 
rameterized  on  the  differences  of  two  consecutive  fractional  parts  from  the  clock 
valuation  of  the  starting  state.  When  we  consider  a  general  source  zone,  we  leave 
unspecified  these  parameters  and  the  above  construction  reduces  the  optimal-run 
problem  for  weighted  timed  automata  to  a  parametric  shortest- path  problem  in 
directed  graphs.  We  give  a  fix-point  computation  algorithm  to  solve  this  prob¬ 
lem,  so  obtaining  a  doubly-exponential  time  algorithm  solving  the  optimal-run 
problem.  In  case  the  input  automaton  has  only  one  clock  variable,  this  result  can 
be  improved  to  a  single  exponential  by  adapting  to  our  case  the  algorithm  given 
in  [K081,YT091]  for  solving  a  particular  case  of  parametric  shortest-path  prob¬ 
lem.  In  case  the  source  zone  is  a  singleton  we  substitutes  the  parameters  with 
the  actual  values  from  the  starting  state,  and  thus  our  optimization  problem 
is  reduced  to  a  standard  shortest-path  problem.  Using  Dijkstra’s  algorithm,  we 
obtain  an  exponential  time  algorithm  for  the  single-source  optimal-run  problem. 

The  optimal-reachability  problem  is  strictly  related  to  other  decision  prob¬ 
lems,  and  in  particular  to  the  problem  of  synthesizing  an  optimal  controller. 
The  optimal- control  synthesis  problem  can  be  informally  stated  as  the  prob¬ 
lem  of  designing  a  control  which  is  able  to  drive,  at  a  minimum  cost,  the  sys¬ 
tem  into  a  given  target  zone.  In  the  literature,  control  synthesis  problems  have 
been  considered  in  the  context  of  discrete  automata  [Chu62,Tho95],  timed  au¬ 
tomata  [AMP95,MPS95,AM99],  linear  hybrid  automata  [WT97],  and  general 
hybrid  systems  [LTS99,SPS00].  The  design  of  an  optimal  control  for  hybrid  sys¬ 
tems  is  not  trivial  and  in  general  is  undecidable.  The  approach  presented  in 
this  paper,  can  be  adapted  to  solve  the  optimal-control  synthesis  problem  for 
weighted  timed  automata.  We  observe  that  this  generalizes  the  results  obtained 
in  [AM99]  on  the  synthesis  of  a  time-optimal  controller  for  a  timed  automaton. 

The  rest  of  the  paper  is  organized  as  follows.  In  section  2,  we  define  the 
optimal-run  problems  and  we  give  some  examples.  In  section  3,  we  introduce 
a  graph  construction  to  reduce  the  optimal-run  problems  to  the  corresponding 
shortest-path  problems  in  directed  graphs.  In  section  4,  we  present  our  solutions 
to  the  single-source  optimal-run  problem  and  to  the  general  case. 
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2  Preliminaries 

In  this  section  we  define  the  single-source  and  the  parametric  optimal-run  prob¬ 
lems.  We  start  introducing  some  notation  and  the  definition  of  timed  automaton. 

Given  a  set  C  of  n  variables,  a  A:-zone  is  a  subset  of  that  can  be  obtained 
as  a  boolean  combination  of  inequalities  of  the  form  x<y-\rC^x<y-\-c,  x<c^ 
and  X  <  c  where  x,y  G  C  and  c  G  {0, 1, . . . ,  /b}.  We  denote  by  TRUE  the  clock 
constraint  which  is  true  for  any  clock  values.  We  denote  by  Z{C)  the  set  of  all 
the  fc-zones,  for  all  /c  G  N.  A  function  A  :  R""  — >  R^  is  called  a  reset  function  if 
it  is  equal  to  the  identity  on  some  of  the  coordinates  and  zero  on  the  others.  We 
denote  by  An  the  set  of  all  reset  functions  over  R’^.  A  timed  automaton^  A  is  a 
tuple  (Q,  C,  zA,  Inv)  where: 

“-  Q  is  a  finite  set  of  locations; 

—  C  is  a  finite  set  of  n  clock  variables; 

—  ZA  is  a  finite  subset  of  Q  x  Z{C)  x  An  x  Q\ 

—  Inv  :  Q  — >  Z{C)  maps  each  location  q  to  its  invariant  Inv(g). 

A  state  is  a  tuple  (g,  u)  where  q  £  Q  and  u  £  We  denote  hy  S  —  QxW^  the 
set  of  states  for  A.  A  discrete  step  is  (g,  i')~^{q',  jy')  where  e  =  (g,  5,  A,  q')  £  A, 
u  satisfies  5,  v'  =  A(i/),  and  ly'  satisfies  Inv(^').  A  time  step  is 
where  i/'  =  i/  -h  t,  t  >  0,  and  ly  +  t'  satisfies  Inv(g)  for  all  0  <  t'  <  t. 

A  step  is  {q,iy)^iq\i'')  where  {q,iy)  and  {q,i>")-e^{q' for 

some  ly"  £  R^,  that  is  a  transition  e  taken  after  spending  some  time  t  in 
the  current  location.  A  run  r  of  a  timed  automaton  A  is  a  finite  sequence 

{qoi^o)~^  {qk-i^^k~i)~^  (Qki^k)'  AVe  say  that  r  starts  at 

(go,  and  ends  at  (g^,  The  definition  of  r  allows  time  to  be  spent  after  tak¬ 
ing  the  last  transition  e^-i.  A  weighted  timed  automaton  is  a  timed  automaton 
A  with  the  following  cost  functions: 

—  Js  :  A  — >  N  {switch  cost)^  and 

—  Jd  ‘  Q  — >  N  {duration  cost). 

Given  a  run  r  of  A  and  cost  functions  Js,  and  Jd,  we  associate  costs  to  r  as 
follows: 

—  Jsir)  =  Ei=i  and 

—  Jd{r)  =  Y!lZoU-  Jd{qi)- 

The  total  cost  associated  to  a  run  r  is  then  J(r)  =  J s{'f')  +  AVe  are  inter¬ 

ested  in  determining  optimal-cost  runs  for  a  timed  automaton.  In  the  following 
examples  we  informally  introduce  some  notions  that  we  will  formalize  in  the  rest 
of  the  section. 


^  The  standard  definition  of  timed  automata  requires  also  an  acceptance  condition 
and  a  symbol  alphabet.  Since  we  are  not  interested  in  studying  languages  accepted 
by  timed  automata  we  omit  these  features  here. 
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Example  1.  Consider  the  timed  automaton  defined  in  Figure  1  such  that  Jd{0)  = 
3,  =  1,  and  the  switch  costs  are  all  1.  Suppose  that  we  start  from  state 

5  =  {OjX,y)  ioT  0  <  x,y  <  2  and  we  want  to  reach  a  state  in  location  2. 
Possible  minimal-cost  runs  from  s  to  a  state  s'  —  {2,x\y')  are  either  ri  = 

(0,a:,s/)4T>  (l,a:i,2/i)-^  (2, x  +  2  -  j/, 2),  or  r2  =  (0, x, ?/)-||>  (2,2,y  +  2-x) 
for  ta  —  (2  —  x)  (obviously,  staying  in  location  2  longer  might  only  increase  the 
overall  cost).  According  to  the  cost  function  J,  the  cost  of  is  J3{ri)-\-Jd{ri)  ~ 
2+3ti+(2~2/-ti)  -  4-2/+2ti  and  the  cost  of  r2  is  Js{r2)+Jd{r2)  ^  l+3(2~x)  = 
7  — 3x.  Clearly,  J{ri)  is  minimized  when  ti  =  0,  that  is  the  transition  from  0  to  1 
is  taken  immediately.  Moreover,  assuming  ti  —  0,  J{ri)  <  J{r2)  if  y  >  3(a:  -  1), 
and  J(ri)  >  J(r2),  otherwise.  Thus,  a  minimal-cost  run  from  s  to  a  state  in 
location  2  depends  on  the  clock  valuation  of  state  s. 


Fig.  1.  A  timed  automaton  with  more  than  an  optimal  run  from  a  same  location. 


Example  2.  Consider  the  timed  automaton  defined  in  Figure  2  such  that  Jd(0)  = 
1}  *^^(1)  =  2,  and  the  switch  costs  are  all  1.  Suppose  that  we  start  from 
state  s  =  (Ojx)  for  0  <  a:  <  2  and  we  want  to  reach  a  state  in  location 
2.  Possible  minimal-cost  runs  from  s  to  a  state  s'  =  {2,x')  are  given  by 

t  tf 

n  =  (0,x)-^  (2,2).  Notice  that  is  a  run  parameterized  by  t, 

where  t  is  the  time  at  which  the  first  edge  is  taken.  Thus  J{rt)  =  J d{f't)  = 
2+t  +  2(2  — t  — x)  =  6  —  t~2x.  Hence  the  cost  of  rt  is  minimized  if  t  is  maximized. 
Since  t  <  {2  —  x)  must  hold,  the  optimal  cost  for  a  run  starting  at  s  is  (4  —  x), 
but  none  of  the  runs  starting  at  s  has  such  a  cost.  In  fact,  for  any  actual  run  Vt 
there  exists  a  $  >  0  such  that  t  —  {2-x-^),  and  J(rt)  =  (4-a:-|-^).  Vice-versa, 
for  any  $  >  0  there  exists  a  run  r  such  that  J{r)  =  (4  —  x  d-  ^).  Clearly,  there 
is  not  a  minimal-cost  run  but  we  can  determine  a  run  whose  cost  is  arbitrarily 
close  to  the  optimal  one. 

Now  we  formalize  the  notion  of  optimal  cost,  optimal  run,  and  approximation 
of  an  optimal  run.  Given  a  timed  automaton  A,  a  state  s,  and  a  target  zone  T, 
an  optimal  cost  for  a  run  from  s  to  T  is  a  J*  such  that  J*  <  J{r)  for  any  run  r 
from  s  to  a  state  in  T,  and  for  any  $  >  0  there  is  a  run  r  such  that  J{r)  < 
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Fig.  2.  A  timed  automaton  with  no  optimal  runs  from  a  location. 


If  there  exists  a  run  r*  such  that  J(r*)  =  J*,  then  r*  is  said  to  be  an  optimal 
run.  As  shown  in  Example  2,  sometimes  an  optimal  run  from  a  state  s  to  a 
target  zone  T  does  not  exist.  In  these  cases,  we  are  interested  in  a  family  R 
of  runs  such  that  all  the  runs  coincide  on  the  sequence  of  switches  and  for  any 
^  G  M-i-  there  exists  a  run  r  G  i?  such  that  J{r)  <  J*  where  J*  is  the  optimal 
cost  over  all  runs  from  s  to  T.  That  is  we  can  determine  a  sequence  of  runs  in 
R  whose  costs  are  arbitrarily  close  to  J*.  We  call  such  a  family  of  runs  R  an 
approximation  of  an  optimal  run.  Given  a  timed  automaton  A,  a  source  zone  5, 
and  a  target  zone  T,  we  consider  the  problem  of  determining  an  optimal  run  from 
a  given  state  s  G  S'  to  T,  if  one  exists,  or  an  approximation  of  an  optimal  run, 
otherwise.  We  call  this  problem  a  single-source  optimal-run  problem.  We  also 
consider  a  more  general  problem,  a  zone  optimal-run  problem ,  defined  as  the 
problem  of  determining  a  symbolic  representation  of  the  solution  to  the  single¬ 
source  optimal-run  problem  for  all  states  in  S.  In  Example  1,  if  we  consider  as 
target  region  all  the  states  in  location  2  and  as  only  source  state  (0, 0,0),  then  a 
solution  to  the  corresponding  instance  of  the  single-source  optimal-run  problem 
is  ri  with  ti  =  0.  As  observed  in  Example  1,  if  we  consider  as  source  zone  the  set 
of  states  (0,  x,y)  such  that  0  <  a:,y  <  1,  then  the  solution  of  the  corresponding 
instance  of  the  zone  optimal-run  problem  is  ri  with  =  0  if  y  >  3(ar  —  1),  and 
r2,  otherwise. 

We  end  this  section  with  an  example  on  an  air-traffic  control  problem  that 
we  will  use  subsequently  in  the  paper. 


Example  3.  Consider  the  timed  automaton  in  Figure  3.  It  models  a  scenario  in 
which  two  aircraft  send  a  landing  request  to  an  airport,  and  our  goal  is  to  allow 
both  the  aircraft  to  land  safely  and  at  minimum  cost.  Safety  requires  that  only 
one  aircraft  at  a  time  must  be  acknowledged  for  landing,  thus  there  are  two 
possible  choices:  aircraft  1  waits  for  the  landing  of  aircraft  2  to  be  completed, 
or  vice-versa.  There  are  costs  ci  and  C2  to  pay  for  forcing  respectively  aircraft 
1  and  aircraft  2  to  wait.  Moreover,  there  is  also  a  cost,  expressed  by  iCi,  which 
is  related  to  the  time  spent  waiting.  Alternatively,  aircraft  i  can  make,  at  a  cost 
c^,  a  maneuver  that  allows  to  spend  w[  instead  of  Wi  per  each  time  unit.  This 
maneuver  takes  at  least  time  1 .  Since  it  is  realistic  to  reduce  the  time  a  runway 
stays  unused,  we  penalize  this  event  by  a  cost  Cq  per  time  unit.  Finally,  we 
assume  that  the  landing  of  each  aircraft  takes  at  least  time  1  since  the  related 
acknowledgement  was  issued  by  the  control  tower. 
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Fig.  3.  An  air-traffic  control  problem. 


3  The  Graph  Construction 

In  this  section  we  give  the  graph  construction  underlying  the  reduction  of  the 
single-source  optimal-run  problem  to  the  shortest-path  problem  and  the  zone 
optimal-run  problem  to  a  parametric  shortest-path  problem.  The  obtained  graph 
is  a  refinement  of  the  region  automaton  [AD94]  of  a  timed  automaton,  in  the 
sense  that  each  vertex  v  carries  more  information  than  a  region.  This  additional 
information  mainly  concerns  the  sequence  of  resets  needed  to  reach  v  from  a 
starting  vertex,  and  the  construction  preserves  the  transitions  of  the  region  au¬ 
tomaton.  Via  this  construction  we  emphasize  the  states  of  the  timed  automaton 
that  might  be  visited  in  some  optimal  runs.  We  start  by  recalling  the  concepts 
of  labelled  directed  graph  and  region  automaton,  then  we  describe  our  graph 
construction. 

Let  <9  be  a  set  of  real- valued  parameters,  we  denote  by  D  the  set  of  linear 
expressions  over  Q.  Given  an  alphabet  a  D -labelled  directed  graph  G  is  a  pair 
(V,  E),  where  V  is  a  set  of  vertices,  and  E  C  Vx  D  x  V  is  a  set  of  iP-labelled  edges. 

A  path  TT  from  vq  to  Vn  in  G  is  a  sequence  vq  vi  Vn 

such  that  Vi^i  Vi  ^  E  ioi  i  —  1, . . .  ^n.  For  a  path  tt,  the  cost  of  tt  is  given  by 
fi-  ^  path  TT  from  v  to  v'  is  a  shortest  path  if  tt  is  the  path  with  minimum 
cost  among  those  connecting  v  to  v' .  Notice  that  varying  the  values  of  parameters 
in  &  the  shortest  path  of  a  graph  may  change,  that  is  to  different  valuations  of 
parameters  may  correspond  different  sets  of  shortest  paths  in  the  graph. 

Consider  now  a  timed  automaton  A.  By  definition  its  set  of  states  is  infinite. 
However,  they  can  be  partitioned  in  a  finite  number  of  equivalence  classes,  called 
regions^  which  are  defined  by  a  location  and  a  clock  region.  Denoted  by  the 
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largest  constant  in  clock  constraints  involving  the  clock  variable  a:,  a  clock  region 
is  described  by: 

-  a  constraint  of  type  c  -  1  <  a:  <  c,  x  >  c^;,  or  x  =  c  for  each  clock  variable 
X  and  c  < 

—  the  ordering  of  the  fractional  parts  of  the  clock  variables  x  such  that  x  <  Cx- 

Thus  a  clock  region  denotes  a  set  of  clock  valuations.  Given  a  clock  valuation 
I/,  [u]  denotes  the  clock  region  containing  u.  A  state  {q,  u)  belongs  to  a  region 
{q',a)  ii  q  =  q'  and  v  £  a.  k  clock  region  a  is  said  to  be  open  if  for  any  clock 
variable  x  and  c  <  c^,  x  =  c  does  not  hold  in  a.  Otherwise  a  is  said  to  be 
a  boundary  clock  region.  These  definitions  apply  to  regions  in  an  obvious  way. 
The  key  property  of  this  equivalence,  is  that  all  the  valuations  belonging  to  a 
region  satisfy  the  same  set  of  clock  constraints  from  the  given  timed  automaton. 
Consistently  we  say  that  a  clock  region  a  satisfies  a  constraint  S  if  i/  satisfies  6 
for  any  u  G  a.  A  clock  region  a'  is  said  to  be  a  time-successor  of  a  clock  region 
a  if  and  only  if  for  any  z/  G  a  there  is  a  d  G  such  that  ly-^-d  £  a'.  The  region 
automaton  of  A  is  a  transition  system  defined  by: 

—  the  set  of  states  R{S)  =  {{q,CK)  |  q  E  Q  and  a  is  a  clock  region  for  A}; 

-  the  transition  rules  R{A)  such  that:  {{q,a),  (q\a'))  G  R(A)  if  and  only  if 
(g,  A,  d,  gf')  G  A  and  there  is  a  time-successor  a"  of  a  such  that  a"  satisfies  5 
and  a'  =  [A  Oja". 

We  denote  the  region  automaton  corresponding  to  A  as  R{A).  For  the  sake  of 
simplicity,  in  the  following  when  no  confusion  can  arise  we  refer  to  the  value  of 
a  clock  variable  x  by  x  itself.  With  x  we  denote  the  fractional  part  of  a  clock 
variable  x.  Let  s  =  {q,  u)  be  a  state  of  A  and  (0  x'^  1) 

be  the  ordering  of  the  fractional  parts  of  the  region  containing  a  clock  valuation 
V  (notice  that  is  either  =  or  <).  With  d(s)  =  (?^i, . . .  ,div+i)  we  denote  the 
differences  between  consecutive  values  in  the  above  ordering,  that  is  di  =  x'l, 
=  1  ^  =  2,  ...,A^.  In  the  following  we 

will  use  (di, . . . ,  div+i)  to  denote  these  differences  in  the  starting  state.  The 
graph  we  are  going  to  define  is  parameterized  over  (di, . . . ,  djv+i)-  Moreover, 
for  i,j  <  N,  we  denote  by  I(i,j)  the  set  of  integers  {z, . . . ,  j  —  1},  if  z  <  j,  and 
{L  . . . ,  iV}  U  {1, . . .  -  1},  otherwise. 

The  region  automaton  does  not  carry  enough  information  to  solve  our  op¬ 
timization  problems.  Thus  we  define  a  labelled  directed  graph  whose  vertices 
correspond  to  “sub-states”  of  the  region  automaton.  For  a  given  state  {q,(y') 
of  the  region  automaton,  a  sub-state  (g,  a)  is  such  that  a  is  a  convex  region 
contained  in  a'.  Denoted  by  (0  ^5:^1  x[  ~2  •  •  •  ~/i+i  1)  the  ordering  of  the 

fractional  parts  in  a  clock  region  a',  we  consider  sub-regions  a  of  a'  such  that 
for  some  of  the  which  are  equal  to  <,  the  difference  between  x[_^  and  x'  is 
very  close  to  0.  Thus  we  represent  a  by  a'  and  specifying  in  the  ordering  of  the 
fractional  parts  if  a  <  is  relative  to  a  “small”  difference  (denoted  by  <)  or  to  a 
“large”  difference  (denoted  by  <).  We  call  each  such  sub-region  a  a  boundary 
sub-region.  Intuitively,  the  reason  we  are  interested  in  boundary  sub-regions  is 
that  the  cost  functions  we  consider  are  linear,  and  their  infimum  over  a  given 
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region  is  reached  on  the  boundary.  Thus  optimal  runs  leave  open  regions  from 
states  which  are  arbitrarily  close  to  their  boundaries.  As  a  consequence  optimal 
runs  visit  also  states  characterized  by  having  clocks  values  either  with  arbitrarily 
close  fractional  parts  or  with  fractional  parts  which  reflects  the  starting  state  and 
the  reset  history  of  the  computation.  For  this  reason,  we  add  to  each  boundary 
sub-region  a  tuple  of  indices  (ii, . . . ,  4)  from  {1, . . . ,  n  +  1}  such  that:  k  is  the 
number  of  large  differences  in  the  ordering  of  the  fractional  parts,  ii  corresponds 
to  the  l-th  large  difference  in  the  ordering  of  the  fractional  parts,  and  there  exists 
a  d  G  {1, . . . ,  A:}  such  that  id^h  <  for  /i  =  0, . . . ,  A:  —  1,  where  the  sums 

(d  -h  h  -f- 1)  and  (d  -f  h)  are  modulo  k.  We  call  such  tuples  distance  tuples,  since 
they  are  used  to  store  the  difference  between  two  consecutive  fractional  parts 
when  this  difference  is  “large”  (i.e.,  they  are  not  arbitrarily  close).  We  define  the 
set  of  vertices  V  as  the  set  of  tuples  {q,  a,  (zi, . . . ,  i^))  where  g  is  a  location,  a  is 
a  boundary  sub-region,  and  {ii,..  .,ik)  is  a  distance  tuple  from 
For  a  vertex  {q,  a,  (zi, . . . ,  4)),  the  sum  gives  the  time  to  leave  the 

region  since  this  subregion  is  entered. 

The  set  of  edges  E  contains  three  types  of  edges:  immediate  switches,  time 
edges  and  delayed  switches.  Informally,  immediate  switches  correspond  to  tran¬ 
sitions  taken  in  the  current  state,  time  edges  correspond  to  letting  time  elapse 
until  the  next  region  is  reached,  and  delayed  switches  correspond  to  transitions 
taken  at  the  “beginning”  or  at  the  “end”  of  the  closest  open  region  (this  region 
if  it  is  an  open  region,  the  next  otherwise). 

Given  two  vertices  v  =  {q,a,{ii, . . .  ,ih))  and  v'  =  {q' ,  P,(ji, . . .  ,jk))),  there 

is  an  immediate  switch  v  v'  if  there  exists  a  transition  e  of  R{A)  from  {q,  a') 
to  {q',p'),  where  a'  and  j3'  are  respectively  the  regions  of  R{A)  containing  a  and 
13,  and  the  sequence  {ji, . . .  ,jk)  is  obtained  from  (zi, . . . ,  z^)  by  deleting  all  the 
indices  ii  such  that  all  the  clocks  between  the  l-th  and  the  (/  -1-  l)-th  large 
differences  (in  the  ordering  of  the  fractional  parts  of  a')  are  reset  in  e. 

Consider  a  vertex  v  =  {q,  a,  (ii, . . .  ,ih))  and  let  (0  ~i  •  ~/c  Vk 

1)  be  the  ordering  of  the  fractional  parts  in  a.  If  we  assume  that  a{yk)  + 1  is  not 
larger  than  the  largest  constant  in  the  timing  constraints  involving  pk  (i.e.,  when 
time  elapses  the  first  integer  value  reached  by  pk  is  at  most  this  constant),  we 
add  to  E  a.  time  edge  v  v'  for  v'  —  {q,j3,  {ji,. . .  ,jh'))  where  [3  is  the  closest 
time-successor  of  a  such  that  the  conditions  expressed  by  one  of  the  rows  of  the 
following  Table  1  are  satisfied  (where  (0  p[  ~2  •  •  •  1)  denotes 

the  ordering  of  the  fractional  parts  in  /3,  and  I  =  2, ...  ,k): 


~i 

~2 

~!+i 

in,-- 

■,3h') 

C 

1. 

< 

< 

< 

i2-)  • 

•  >  ^h—l) 

2. 

^  or  = 

< 

= 

< 

(z/i,  Zi,  .  . 

•  '  3  ‘Ih—l) 

Jd{q) 

3. 

< 

< 

= 

< 

(zi,.. 

•  3  'Ih) 

0 

4. 

^  or  = 

< 

< 

(n,-. 

•  I'lh) 

0 

In  the  other  case,  time  edges  are  defined  in  the  same  way  except  for  the 
fact  that  the  clock  pk  does  not  appear  in  the  ordering  of  the  fractional  parts  of 
v'  since  it  has  reached  its  highest  constant.  To  see  an  example  of  a  time  edge, 
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consider  a  vertex  v  =  {q,  ^<x<y<z<\^  (1, 2, 3, 4)).  By  row  1  of  the  above 
table  we  have  a  time  edge  from  v  to  {q,  0<x<y<l^z  =  l,  (4, 2, 3)).  The 
distance  tuple  (4, 2, 3)  captures  the  fact  that  time  {I  -  z)  has  elapsed  and  thus 
the  distance  in  time  from  a:  to  0  is  increased  by  (1  —  2),  the  fractional  part  of  2 
is  now  0,  and  all  the  other  distances  stay  unchanged. 

Given  a  vertex  v  G  V  as  above,  we  add  to  .E  a  delayed  switch  v  — ^  v"  for 

j  si^) 

any  vertex  v"  €  V  such  that  there  exists  an  immediate  switch  v'  v"  and 
c  =  c'  +  Js(e),  where  v'  =  {q,/3,  (ji, . . .  Jh'))  and  /3  is  the  closest  time-successor 
of  a  such  that  the  conditions  expressed  by  one  of  the  rows  of  the  following  Table 
2  are  satisfied  (where  (0  y[  ^2  •  -  ^'k  v'k  1)  denotes  the  ordering  of  the 

fractional  parts  in  /?,  and  I  =  2, ....  k): 
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(ii 5 • • 
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For  a  given  tuple  of  parameters  =  (i^i, . . , ,  we  denote  by  Oa{'^)  the 

E-labelled  directed  graph  (F,  E).  We  recall  that  for  our  purposes  represents  the 
differences  between  the  fractional  parts  of  two  consecutive  clocks  in  the  ordering 
of  the  fractional  parts  in  the  starting  state.  The  construction  of  Ga('^)  is  general 
in  the  sense  that  it  does  not  depend  on  the  particular  source  and  target  zones  of 
the  problem,  but  only  on  the  timed  automaton.  This  allows  us  to  use  it  for  solving 
both  the  single-source  optimal-run  problem  (for  a  fixed  and  the  zone  optimal- 
run  problem  {'d  belongs  to  a  convex  set).  As  an  example  of  application  of  the 
above  construction,  we  discuss  a  fragment  of  the  graph  Ga{'<^)  for  the  weighted 
timed  automaton  modelling  the  air-traffic  control  problem  from  Example  3  (see 
Figure  4).  For  the  sake  of  simplicity,  we  have  marked  with  1, . . . ,  5  the  vertices 
of  Ga('0)  in  Figure  4,  and  we  refer  to  them  by  these  numbers.  Consider  vertex 
1.  Since  in  the  timed  automaton  from  Figure  3  there  is  a  transition  from  Wi  to 
Wj  resetting  clock  xi,  we  have  in  GaW  an  immediate  switch  from  1  to  2.  Edges 
from  1  to  3  and  from  1  to  4  are  delayed  switches  obtained  by  the  same  transition 
above  and  respectively  rows  3  and  4  of  Table  2.  The  edge  from  1  to  5  is  a  time 
edge  and  is  defined  by  row  2  of  Table  1.  Notice  that  for  a  given  state  s  =  {q,  i/),  we 
have  corresponding  vertices  of  Gy4('i9(s))  of  form  {q,  a,  (zi, . . . ,  Zfc)),  where  1/  E  a. 
Moreover,  each  edge  is  labelled  by  the  actual  cost  of  the  corresponding  “activity” 
in  A,  that  is  for  immediate  switches  we  have  just  the  cost  of  the  A  transition, 
for  time  edges  the  cost  of  spending  the  time  upto  the  end  of  the  current  region 
in  the  current  A  location,  and  for  delayed  switches  the  cost  corresponding  to  the 
A  transition  plus  the  cost  for  the  time  spent  in  the  current  location  before  that 
the  transition  is  taken.  We  have  the  following  lemma. 


Lemma  1.  Given  a  timed  automaton  A,  the  size  of  GAi'd)  is  exponential  in  the 
length  of  clock  constraints  of  A. 
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Fig.  4.  A  fragment  of  Ga{'^)  for  the  weighted  timed  automaton  in  Example  3. 


Proof.  In  [AD94]  the  authors  proved  that  the  size  of  the  region  automaton  is 
0(|A|  where  |^(A)|  denotes  the  length  of  the  clock  constraints.  A  sim¬ 

ple  counting  argument  gives  that  the  number  of  ways  to  substitute  <  with  < 
in  the  ordering  of  the  fractional  parts  of  a  clock  region  is  at  most  2^^^  and 
the  number  of  tuples  of  indices  we  use  to  represent  the  relative  differences  be¬ 
tween  the  fractional  parts  is  at  most  n2^.  Thus  the  size  of  is  at  most 

0{\A\  n2^"'+^  and  since  n  =  0(|(5(A.)|),  it  is  exponential  in  the  length  of 

the  clock  constraints. 

4  Optimal-Runs  in  Weighted  Timed  Automata 

4.1  Single-Source  Case 

In  this  section  we  prove  that  the  single-source  optimal-run  problem  in  timed 
automata  can  be  reduced  to  the  shortest  path  problem  in  a  weighted  directed 
graph.  To  see  this  we  introduce  first  some  notation.  Let  sq  be  a  state  {qo^uo) 
of  a  weighted  timed  automaton  A  and  ?9(so)  =  (^^i, . . .  we  denote  by 

g{so)  the  vertex  {go,  oiQ,  ...  Ao, No))  of  <^a(^(so))  such  that  uq  e  oq  and 
ioj  is  the  j-th  largest  distance  in  the  ordering  of  the  fractional  parts  in  Oq- 
Given  a  positive  real  ^  <<  1  and  a  path  tt  =  (^o,  o;o,  {^,i,  •  •  • ,  ^o.ato)) 

{qi,o^i,{ii^i,. . .  Ai,Nr))  ...  {qh,Oih,{ih,i,--‘,ih,Nh))  in  <^a(^(so)),  we 

denote  by  RttH)  the  set  of  runs  of  A  starting  at  sq  and  obtained  by  replac¬ 
ing  with  {qk,J^k)  each  portion  {qj,aj,{ij^i, . . .  ^  ^ 

(g/c,0!/c,(4,i,. .  .,4,ArJ)  of  TT  such  that: 

-  (zj_i,i, . . .  {qj,(Xj,{ij^i,...Aj,Nj))  is  either  an 

immediate  or  a  delayed  switch; 

-  ^ov  I  ~  -  2,  (qhOii,{ii^i, . . .  ,ii,Ni)) 

{q'/+i,o;z+i,  (fr+1,1,  ■  • .  ,fr+i,iVz+i))  is  a  time  edge; 


Cj+I 
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{9k— i-)  ^k—ii  1,1’  ■ '  •  5  ^k—i,Nk-i))  ^  :  '^k,Nk'}}  Cither  an 

immediate  or  a  delayed  switch.  Let  tj  —  r'  +  r"  and  i^j  +  r'  G  afc-i*  In  the 
case  of  an  immediate  switch  r”  =  0,  while  in  the  other  case  r"  is  such  that: 

—  if  the  delayed  switch  is  obtained  by  rows  1  and  2  of  Table  2,  then  G 

ak-i  and  the  largest  fractional  part  in  i/j  +  tj  is  greater  than  (1  ~  Ol 

—  otherwise,  denoted  as  a'  the  time-successor  of  which  is  first  entered 
by  letting  time  elapse  from  a  valuation  in  q;a;_i  ,  it  holds  that  I'j  +  tj  G  a', 
moreover  if  the  delayed  switch  is  obtained  by  rows  4  and  5  of  Table  2, 
the  largest  meaningful  fractional  part  in  Uj  +  tj  is  greater  than  (1  — 
and  if  the  delayed  switch  is  obtained  by  rows  3  and  5  of  Table  2,  the 
smallest  meaningful  fractional  part  in  i^j  +  tj  is  less  than 

-  Cj  is  the  transition  corresponding  to  {qk-i,ak-i,  (4-i,i,  •  •  •  ,ik-i,Nk-i)) 

{Qkl  Oik  7  (^fc,l  ?  •  ■  •  7  ))  • 

In  the  following  we  assume  that  ^  is  a  positive  real  number  such  that  $  <<  1. 
By  the  definition  of  and  following  lemma. 

Lemma  2.  Given  a  timed  automaton  A  and  a  state  s  =  {q,  u)  of  A,  if  tt  is  a 
path  o/G^(?9(s))  from  g{s)  of  cost  c-jr  then  is  a  set  of  runs  of  A  such  that 

for  any  e  >  0  there  exists  an  r  G  R-kH)  such  that  Ctt  <  J{r)  <  Ct^ 

To  complete  our  reduction  we  need  the  following  lemma. 

Lemma  3.  Given  a  run  r  of  A  from  a  state  s  to  a  target  zone  T,  there  exists 
a  path  TT  of  G Aid {s))  from  g{s)  to  a  vertex  corresponding  to  a  state  in  T  such 
that  the  cost  of  n  is  not  larger  than  J(r). 

Proof  The  interesting  case  is  when  transitions  in  r  are  from  states  that  do  not 
belong  to  any  of  the  subregions  encoded  by  G^('i?(s))  vertices.  Assume  that  A 
in  run  r  takes  a  transition  e  from  an  open  region  o;  after  spending  some  time  in 
it,  and  e  is  the  first  transition  in  r  with  this  property.  Clearly,  upto  e,  r  has  a 
corresponding  path  tt  in  G'^('i9(s))  whose  cost  is  not  more  than  J(r).  We  observe 
that  by  definition  there  must  be  two  delayed  transitions  ei  and  62  of  G>i(^9(s)) 
corresponding  respectively  to  the  cases  e  is  taken  as  soon  as  a  is  entered  and 
e  is  taken  just  before  leaving  a.  Moreover,  consider  two  A  runs  ri  and  r2  that 
differ  from  r  only  for  the  fact  that  in  ri  A  takes  e  after  an  arbitrarily  short  time 
spent  in  a,  while  in  r^  A  takes  e  after  an  arbitrarily  short  time  before  leaving 
a.  Clearly,  J(r)  >  min{  J(ri),  J{r2)}  holds.  Thus  we  can  add  to  tt  the  transition 
corresponding  to  the  run  with  the  least  cost  between  ri  and  r2.  Applying 
iteratively  this  argument,  we  determine  a  path  tt  in  GA{'d{s))  of  cost  c  <  J{r). 

As  a  direct  consequence  of  Lemmas  2  and  3,  we  have  the  following  theorems. 


Theorem  1.  Given  a  timed  automaton  A,  a  state  s  of  A,  a  target  zone  T,  tt 
is  a  shortest  path  o/G^('i?(s))  starting  from  g{s)  to  a  vertex  corresponding  to  a 
state  in  T  if  and  only  if  R-k{0  approximation  of  an  optimal  run  of  A  from 

s  to  T. 
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Theorem  2.  Given  a  timed  automaton  A,  a  state  s  of  A,  a  target  zone  T ,  there 
exists  an  optimal  run  of  A  from  s  to  T  if  and  only  if  for  a  shortest  path  tt  of 
from  g{s)  to  a  vertex  corresponding  to  a  state  in  T  there  exists  a  run 
r  G  i?7r{Oj  ^^o.t  the  cost  of  tt  is  equal  to  J{r).  Moreover,  r  is  an  optimal 
run  of  A  from  s  to  T. 

Given  a  timed  automaton  A,  a  source  state  s,  and  a  target  zone  T,  the 
following  algorithm  solves  the  single-source  optimal-run  problem: 

1.  Let  G  be  the  graph  obtained  from  Gyi(i9(s))  by  collapsing  all  the  vertices 
corresponding  to  a  state  in  T  in  a  single  vertex  Vt . 

2.  Solve  the  single-source  shortest-path  problem  on  G  from  g{s). 

3.  Let  TT  be  a  shortest  path  from  Vs  to  Vf  Output^  RniO  ^nd  the  cost  of  tt. 


Theorem  3.  The  single-source  optimal-rxbn  problem  can  be  solved  in  time  ex¬ 
ponential  in  the  size  of  the  timed  automaton. 


4.2  The  Algorithm  for  the  General  Case 

In  this  section  we  consider  the  zone  optimal-run  problem.  We  give  an  exponential 
time  algorithm  to  solve  this  problem  for  timed  automata  with  at  most  1  clock 
and  a  fix-point  algorithm  in  doubly-exponential  time,  for  the  general  case. 

We  start  considering  the  general  case.  Since  we  want  to  solve  the  problem 
of  determining  the  optimal  runs  from  any  state  of  the  source  zone  5  to  a  state 
of  a  target  zone  T,  for  parameters  d  in  we  consider  only  values  given 

hy  -d  =  id{s)  for  a  state  in  s  G  5.  Thus  it  holds  that  -f  . . .  +  '^n+i  =  1 
and  we  can  eliminate  a  parameter  by  the  substitution  =  1  — 

From  now  on,  we  will  assume  that  'd{s)  is  the  tuple  (t9i,  . . .  and  G^(^(s)) 
is  the  graph  obtained  after  the  substitution  =  1  —  The  algo¬ 
rithm  that  we  are  giving,  labels  the  vertices  of  with  sets  of  linear  ex¬ 
pressions  on 'd  =  (?9i, . . .  The  meaning  of  these  expressions  is  that  given 

a  state  s  G  5  the  minimum  over  these  expressions  gives  the  optimal  cost  of 
a  run  from  s.  An  expression  is  a  first-degree  polynomial  in  . . . ,  and 
(1  -  Z)i=i  ^0  with  integer  coefficients.  That  is,  an  expression  has  the  form 

f{d)  =  oo  +  aidi  +  . . .  +  +  aiv+i(l  -  where  ao, . . . ,  ciN+i  are 

nonnegative  integer  constants.  We  denote  expressions  by  (AT  +  2)- tuples  of  co¬ 
efficients  and  write  (oq,  . . . ,  oat+i)  for  the  above  expression  f{d).  We  denote  by 
-<  the  natural  extension  to  tuples  of  the  total  ordering  <  over  reals.  Moreover, 
let  /,/'  be  two  expressions,  and  v,v'  be  two  vertices  of  GA{d),  (/,u)  {f\v') 

if  and  only  if  /  -<  f'.  A  set  X  of  tuples  of  type  (/,  u),  for  an  expression  /  and  a 
vertex  v,  is  said  to  be  minimized  (with  respect  to  -<)  if  for  any  {f,v),{f',v')  G  X, 
(/,  u)  and  {f'jv')  are  not  comparable  with  respect  to 

^  This  step  needs  a  further  refinement  to  distinguish  between  an  approximate  solution 
and  an  optimal  solution.  It  is  not  entirely  straightforward,  but  it  can  be  handled  at 
the  same  complexity.  We  defer  the  reader  to  the  full  version  of  the  paper. 
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The  algorithm  we  present  computes  a  labelling  function  I  that  maps  any 
vertex  u  of  to  a  minimized  set  of  pairs  {f,v)  for  which  there  exist  a  path 

TT  and  a  state  s  6  5  such  that: 

-  TT  is  a  shortest  path  of  from  u  to  a  vertex  corresponding  to  T, 

—  the  first  edge  e  of  tt  connects  u  to  v,  and 

—  the  cost  of  TT  is  given  by  /(^9(s)). 

We  can  summarize  our  algorithm  in  the  following  steps: 

1,  Initialize  I  by  assigning  l{u)  =  {(0, . . .  ,0,u)}  for  u  corresponding  to  a  state 
in  T,  and  l{u)  =0  for  all  remaining  vertices. 

2,  repeat 

I'  I  ^  Update(I’) 
until  V  =  I 

3,  Output  L 

We  just  need  to  specify  the  function  Update.  Consider  an  edge  e  a  vertex 
u  =  {q-,OL-,  (ii, . . .  We  have  the  following  cases: 

—  e  is  an  immediate  switch  from  u  to  v\  for  (ao, . . . ,  ajv+i?  ^^0  ^  define 

(aj), . . . ,  v)  such  that  ttg  =  ao  +  Ce,  and  for  z  =  1, . . . ,  (AT  +  1), 

where  Cg  is  the  cost  of  e; 

-  e  is  a  time  edge  from  u  to  v:  for  any  (ao, . . . ,  Uiv+i,  ^0  ^  define 

(oq,  . . . ,  r;)  such  that  if  e  is  obtained  by  rows  1  and  2  of  Table  1  and 

1  e  I{ih,h),  then  a •  =  +  Jdiq),  otherwise  a[  =  a^; 

-  the  edge  e  is  a  delayed  switch  from  u  to  v:  for  any  (ao, .  • .  ,aN+i,v')  € 

define  (af,, . . . ,  z;)  such  that  if  e  is  obtained  by  rows  1,  2  and  4  of  Table 

2  and  i  €  I{ih^  ^i),  then  +  Jdiq),  otherwise  =  Ui. 

Let  l"{u)  be  the  set  of  all  the  tuples  generated  for  u.  After  executing  I  <— 
Update(^'),  l{u)  contains  the  set  obtained  deleting  from  l'(u)  U  l"{u)  all  the 
tuples  (f^v)  such  that  f'  ^  f  for  some  {f\v')  €  l'(u)  U  l"(u).  Moreover,  once 
the  function  I  is  output,  it  is  easy  to  determine  the  optimal  cost  and  generate 
the  corresponding  solution  from  I  and  the  graph  GaW,  given  We  observe 
that  each  of  the  tuples  (f^v)  belonging  to  l(u)  corresponds  to  a  path  from  u  to 
a  target  vertex.  Thus  the  cardinality  of  l{u)  is  bounded  above  by  the  number  of 
simple  paths  in  C^('i9).  Hence  we  have  the  following  theorem. 

Theorem  4.  The  zone  optimal-run  problem  can  he  solved  in  doubly- exponential 
time. 

If  we  restrict  to  timed  automata  with  just  one  clock  variable,  it  is  possible  to 
solve  the  zone  optimal-run  problem  in  singly  exponential  time.  We  consider  the 
algorithm  given  in  [K081,YT091]  to  solve  a  particular  shortest-path  problem 
with  only  a  parameter 'd  and  edge  costs  given  by  (c  ~  z?),  for  constants  c.  This 
algorithm  runs  in  polynomial  time  and  can  be  modified  in  order  to  obtain  a 
polynomial  time  algorithm  to  solve  the  parametric  shortest-path  problem  with 
edge  costs  given  by  a  first-degree  polynomial  of  z?  (??  G  [0, 1]). 

Theorem  5.  The  zone  optimal-run  problem  for  automata  with  one  clock  vari¬ 
able  can  be  solved  in  exponential  time. 
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Abstract.  Reach  set  computations  are  of  fundamental  importance  in 
control  theory.  We  consider  the  reach  set  problem  for  open-loop  systems 
described  by  parametric  inhomogeneous  linear  differential  systems  and 
use  real  quantifier  elimination  methods  to  get  exact  and  approximate 
solutions.  The  method  employs  a  reduction  of  the  forward  and  back¬ 
ward  reach  set  and  control  parameter  set  problems  to  the  transcendental 
implicitization  problems  for  the  components  of  special  solutions  of  sim¬ 
pler  non-parametric  systems.  For  simple  elementary  functions  we  give  an 
exact  calculation  of  the  cases  where  exact  semialgebraic  transcendental 
implicitization  is  possible.  For  the  negative  cases  we  provide  approximate 
alternating  using  discrete  point  checking  or  safe  estimations  of  reach  sets 
and  control  parameter  sets.  Examples  are  computed  using  the  REDLOG 
and  QEPCAD  packages. 


1  Introduction 

Today  integrated  systems  which  combine  physical  processes  with  information 
systems  {i.e.  digital  programs)  are  in  great  demand.  In  fact  complex  systems 
which  have  been  designed  recently  incorporate  both  differential  equations  to 
model  the  continuous  behavior  and  discrete  event  systems  to  model  instanta¬ 
neous  state  changes  in  response  to  events.  Systems  that  are  finite  state  machines 
with  differential  equations  at  each  discrete  state  are  called  Hybrid  Systems. 

A  lot  of  research  effort  has  been  devoted  to  develop  mathematical  models, 
specification  formalisms,  analysis/design/control  methods  and  tools  to  help  con¬ 
trol  engineers  in  building  such  systems  (see  [18,30,26]),  Most  of  the  applications 
of  hybrid  systems  are  safety  critical.  Safety  is  usually  encoded  as  avoidance  of  an 
undesirable  region  of  the  state  space.  Consequently,  the  most  important  prob¬ 
lems  for  analyzing  hybrid  systems  are  verification  problems;  these  are  essentially 
reachability  problems,  that  ask  whether  trajectories  of  the  hybrid  systems  reach 
certain  undesirable  (unsafe)  regions  from  an  initial  region. 

Computing  the  reach  set  of  hybrid  systems  is  difficult  because  hybrid  systems 
have  an  infinite  state  space.  Due  to  the  difficulty  of  computing  the  reach  set  for 
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systems  of  differential  equations,  formal  verification  methods  and  tools  for  hybrid 
systems  have  been  developed  [2,17].  These  methods  and  tools,  however,  can  deal 
with  only  very  simple  continuous  models  as,  e.p.  i:  =  1,  Ax  =  6.  What  is  actually 
required  is  to  handle  hybrid  systems  with  more  complicated  continuous  parts. 

Decidability  of  reachability  problem  for  hybrid  systems  with  linear  differen¬ 
tial  equation  of  the  form  y  =  Ay^-Bu\s  discussed  in  [23,24].  This  is  a  significant 
class  of  linear  differential  equations  that  is  widely  used  in  linear  control  theory. 
The  results  are  based  on  the  notion  of  Vminimality”  [16]  from  model  theory 
and  “quantifier  elimination”  [11].  0-minimality  is  used  to  define  a  class  of  hy¬ 
brid  systems  “o-minimal  hybrid  systems”  and  it  is  shown  that  all  o-minimal 
hybrid  systems  admit  finite  bisimulations  in  [22].  To  make  the  bisimulation  al¬ 
gorithm  computationally  feasible,  they  utilize  mathematical  logic,  in  particular, 
real  quantifier  elimination,  as  main  tool  to  represent  and  manipulate  sets  sym¬ 
bolically.  Since  quantifier  elimination,  in  general,  is  possible  for  the  polynomial 
theory  of  reals  [11],  they  have  found  subclasses  of  o-minimal  hybrid  systems  that 
are  definable  in  the  theory. 

Remark:  There  are  many  results  that  apply  quantifier  elimination  to  control  theory  [6 , 
15,19,4].  In  [28,3]  quantifier  elimination  is  used  for  verification  problems  (reachability 
and  observability  problems)  of  discrete-time  polynomial  systems. 

In  this  paper  we  study  in  particular  reach  set  problems  for  continuous  open- 
loop  systems  described  by  parametric  systems  of  linear  differential  equations  [21]. 
Roughly  speaking  reach  set  problems  are  concerned  with  the  relations  between 
possible  values  of  the  state  variables  at  some  initial  time  to  and  the  corresponding 
values  at  later  points  in  time.  The  specific  problems  studied  in  this  paper  are 
the  following: 

1.  Fix  a  set  M  of  values  of  the  state  variables  at  to;  what  are  the  possible 
corresponding  values  at  later  points  t  in  time  (up  to  some  bound  ti  or  oo). 
{Forward  reach  set) 

2.  Fix  a  set  iV  of  “safe”  values  of  the  state  variables.  Find  a  set  M  as  large 
as  possible  of  initial  values  of  the  state  variables  at  time  to  that  guarantees 
that  the  values  of  the  state  variables  will  for  all  later  time  points  t  (up  to 
some  bound  ti  or  oo)  remain  inside  N.  {Backward  reach  set) 

3.  Fix  a  set  M  of  values  of  the  state  variables  at  to  and  a  set  N  of  “safe” 
values  of  the  state  variables.  Find  a  set  P  as  large  as  possible  of  the  control 
parameters  such  that  all  state  variables  with  initial  values  at  to  in  M  will 
have  values  in  N  for  all  later  time  points  t  (up  to  some  bound  ti  or  oo.) 
{Control  parameter  set) 

Our  main  tool  is  the  method  of  real  quantifier  elimination  in  computer  algebra. 
This  approach  was  introduced  into  reach  set  computations  in  [29].  In  a  series  of 
papers  they  showed  how  to  get  exact  solutions  of  the  forward  reach  set  problem 
for  certain  homogeneous  linear  differential  systems  of  special  type  with  constant 
coefficients  [23]  and  for  associated  inhomogeneous  systems  with  very  special 
right  hand  side  [24].  The  exact  solutions  are  always  obtained  as  semialgebraic 
sets  described  by  a  boolean  combination  of  polynomial  inequalities. 
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Here  we  extend  this  ad  hoc  approach  for  special  types  of  differential  systems 
to  a  systematic  study  of  the  type  of  results  obtainable  by  an  approach  via  real 
quantifier  elimination.  By  reducing  the  approach  to  its  bare  essentials,  we  obtain 
a  much  wider  systematic  framework  applicable  to  a  considerably  larger  class 
of  systems.  The  main  observation  is  that  all  the  problems  mentioned  above 
can  be  reduced  by  exact  symbolic  algorithms  to  an  implicitization  problem  for 
certain  basic  transcendental  functions  associated  with  the  given  system.  Exact 
solutions  for  implicitization  problems  with  rational  par ametrizat ions  are  well- 
known  [8,27].  Here  we  deal  with  the  corresponding  problem  for  transcendental 
parametrizations  that  has  been  studied  only  for  special  cases  e.g.  in  [13,20]. 

Our  main  results  are  as  follows:  We  associate  with  every  parametric  linear 
system  of  differential  equations  y  =  A{t)y  +  6(t,r)  a  finite  system  F  of  basic 
functions.  Then  for  semialgebraic  sets  M,N  all  three  problems  can  be  solved 
exactly  by  real  quantifier  elimination  relative  to  the  implicitization  problem  for 
the  components  of  the  functions  in  F.  Moreover  the  discrete  point  version  of 
these  problems  require  only  finitely  many  evaluations  of  functions  in  F.  We 
prove  a  theorem  that  determines  the  exact  classes  of  vector- valued  functions  of 
the  kind  arising  in  linear  differential  systems  with  constant  coefficients,  where 
exact  semialgebraic  implicitization  is  possible.  As  a  corollary  we  obtain  the  exact 
limitations  of  the  approach  of  [23,24]  for  linear  differential  systems  with  constant 
coefficients  and  special  right  hand  sides. 

We  propose  several  ways  to  overcome  these  limitations  by  approximate  com¬ 
putations:  One  way  is  to  compute  exact  reach  sets  at  a  finite  selection  of  discrete 
time  points.  This  is  always  possible  and  practically  quite  efficient,  but  may  lead 
to  underestimation  of  the  true  forward  reach  set,  depending  on  the  selection  of 
time  points.  Another  approach  separates  the  common  time  variable  into  differ¬ 
ent  time  variables.  This  leads  to  an  overestimation  in  the  implicitization  problem 
resulting  in  an  overestimation  of  the  forward  reach  set  and  an  underestimation 
of  the  backward  reach  set  and  the  control  parameter  set:  So  all  three  approxi¬ 
mations  are  on  the  safe  side. 

We  illustrate  some  problems  and  solution  methods  by  examples  computed 
in  the  REDLOG-package  of  REDUCE  [14]  and  QEPCAD  [12].  We  expect  that  our 
results  can  be  extended  to  the  hybrid  systems  with  linear  continuous  parts. 

2  Reach  Sets  and  Transcendental  Implicitization  Problem 

2.1  Problem  Statement 

We  consider  parametric  inhomogeneous  systems  S  of  linear  differential  systems 
of  the  form  y  =  A{t)y  +  h{t,r)  with  an  n  x  n  matrix  A{t)  of  real  continuous 
functions  aij{t)  and  a  vector- valued  real  continuous  function  b{t,r)  defined  on 
some  interval  /.  The  inhomogeneous  part  is  assumed  to  be  a  linear  combination 
b{t,r)  =  with  continuous  functions  gi  :  I  — >  and  real  parame¬ 

ters  Vi.  Such  a  system  can  be  viewed  as  an  continuous  open-loop  control  system 
with  control  parameters  r  =  (ri, . . .  ,rfc).  Let  M  be  some  subset  of  and  fix 
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an  initial  time  point  to  E  I  :  Then  we  denote  the  set  of  all  solution  functions 
/  :  I  — >  of  the  given  system  with  parameters  r  =  {ri,..  ,^rk),  by  and 

the  set  of  all  solution  functions  f  e  Fr  with  initial  value  f{to)GMhyF  =  FM,r- 
We  consider  the  following  forward  reach  set  problems: 

discrete  reach  sets  Compute  for  finitely  many  time  points  <  . . .  <  in  / 
the  union  of  the  sets  {f{ti)  \  f  G  FM,r}- 
bounded  reach  set  Compute  for  a  given  time  ti  >  to  m  I  the  set  {f{t)  |  /  e 

unbounded  reach  set  Suppose  I  D  [^0,00),  and  compute  the  set  {f{t)  \  f  G 
FM,r,to  <  t}. 

All  computations  should  be  performed  in  explicit  dependence  on  the  control 
parameters  r.  Any  solution  of  the  discrete  reach  sets  problem  yields  an  lower 
estimate  for  the  sets  to  be  computed  in  the  bounded  and  unbounded  reach  set 
problems. 

Of  equal  interest  are  the  corresponding  “backward”  reach  set  problems  that 
are  a  kind  of  “dual”  to  the  corresponding  “forward”  problems. 

Some  backward  reach  set  problems  are  as  follows:  Let  N  he  &  subset  of 

backward  discrete  reach  sets  Compute  for  finitely  many  time  points  ti  < 
. . .  <  fm  in  /  the  sets  {f{to)  \  f{ti), . . . ,  f{tm)  G  N}. 
backward  bounded  reach  set  Compute  for  a  given  time  ti  >  to  in  I  the  set 
{/(^o)  I  f{i)  C  N  for  all  to  <  ^  < 

backward  unbounded  reach  set  Suppose  I  D  [to,  00),  and  compute  the  set 
{/(^o)  I  /(t)  6  N  for  all  to  <  t}. 

From  the  viewpoint  of  control  theory  these  problems  have  still  other  vari¬ 
ants  concerning  the  determination  of  suitable  control  parameter  values  r  ~ 
(ri, . . . ,  rfc).  Let  M  as  before  be  a  subset  of  R”,  and  let  N  be  another  subset  of 
M”.  Then  we  have  the  following  natural  control  parameter  set  problems: 

discrete  point  control  Compute  for  finitely  many  time  points  ti  <  . , .  <  t^ 
in  I  the  set  {r  G  |  f{ti)  G  N  for  all  /  G  Fm,t,  1  <  ^  < 'm}. 
bounded  interval  control  Compute  for  a  given  time  ti  >  to  in  I  the  set 
{r  G  R^  I  f{t)  G  N  for  all  /  G  FM,r,to  <t  <  ti}. 
unbounded  interval  control  Suppose  I  D  [to,oc),  and  compute  the  set  {r  G 
R^  I  f{t)  G  N  for  all  /  G  FM,r,  <  i}* 

In  order  to  make  these  problems  mathematically  precise,  we  need  to  specify 
the  way  in  which  the  input  sets  M  and  N,  and  the  output  sets  should  be  de¬ 
scribed.  For  an  approach  using  symbolic  computations  it  is  natural  to  consider 
semialgebraic  sets  as  possible  inputs.  These  are  subsets  of  R^  described  by  a 
boolean  combination  (p{xi, . . . ,  Xn)  of  real  polynomial  inequalities.  If  in  addition 
all  the  polynomials  involved  in  (p{xi, . . .  ,Xn)  are  linear,  then  the  set  described 
by  (p  is  called  semilinear  [16,32]. 

Our  goal  is  to  solve  the  forward  and  backward  reach  set  and  control  parame¬ 
ter  set  problems  for  semialgebraic  input  sets  as  far  as  possible  with  descriptions 
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of  semialgebraic  sets  as  outputs.  This,  however,  is  not  always  possible.  Hence  we 
consider  also  the  computation  of  overestimating  the  forward  reach  sets  and  un¬ 
derestimating  the  backward  reach  set  and  the  control  parameter  sets  by  suitable 
semialgebraic  sets. 

Our  main  tool  will  be  a  reduction  of  reach  set  and  control  parameter  set 
computations  to  corresponding  implicitization  problems  for  a  fixed  finite  sys¬ 
tem  of  functions  associated  with  5,  namely  a  fundamental  system  /i, . .  ■ ,  /n  for 
the  homogeneous  system  5o  associated  with  5,  and  special  solutions  hi  of  the 
parameter-free  inhomogeneous  system  Si  given  hy  y  =  A{t)y-\-gi{t)  for  1  <  i  <  k. 
We  refer  to  {/i, . . . ,  /n,  hi,...,  hk}  as  a  system  of  basic  functions  for  S. 

Implicitization  problems  for  rational  parametrizations  of  algebraic  varieties 
have  been  widely  considered  in  computer  algebra  [8,27].  Here  we  have  to  study 
the  corresponding  problem  for  the  vector- valued  functions  /i,  •  •  • ,  /n,  hi,  ...,hk, 
arising  from  the  system  S.  As  these  functions  will  in  general  be  transcendental, 
we  refer  to  these  problems  as  transcendental  implicitization  problems. 

More  precisely,  we  consider  the  following  transcendental  implicitization  prob¬ 
lems  for  given  functions  fi  :  I  — >  for  1  <  z  <  h  : 

discrete  points  implicitization  Compute  for  finitely  many  time  points  fi  < 
...<  in  7  the  values  (fiiU), . . .  fkiU)),  regarded  as  points  in  R^^. 
bounded  implicitization  Compute  for  a  given  time  ti  >  to  in  I  the  set 
{(fi{t),...fk{t))^R^^\to<t<ti}. 
unbounded  implicitization  Suppose  I  3  [to,oo),  and  compute  the  set 

{(/iW,  ...AW)€R"Mio<0- 

The  first  problem  amounts  to  simple  evaluations  of  the  given  functions.  No¬ 
tice  that  the  unbounded  and  bounded  implicitization  problem  for  a  single  solu¬ 
tion  of  the  differential  system  S  is  in  fact  a  special  case  of  the  unbounded  and 
bounded  forward  reach  set  problem  for  S,  respectively,  namely  for  the  case  of  a 
singleton  set  M. 

2.2  Reduction  to  Implicitization  Problems 

Next  we  show  that  all  reach  set  computations  and  control  parameter  set  com¬ 
putations  listed  above  can  for  semialgebraic  input  sets  M,N  he  reduced  in  an 
exact  symbolic  way  to  one  of  these  implicitization  problems.  All  these  reductions 
require  real  quantifier  elimination  as  fundamental  tool.  For  the  case  of  discrete 
points  forward  and  backward  reach  set  and  control  parameter  set  and  semilinear 
input  sets  M,N  we  find  moreover  that  the  output  sets  are  also  semilinear. 

Let  cp{xi,...,Xn)  and  ipixi, . . .  ,Xn)  be  quantifier-free  formulas  describing 
the  semialgebraic  input  sets  M  and  N,  respectively.  Let  y  ~  Ay  b{t,r)  with 
be  a  parametric  linear  system  S  with  control  parameter  ri. 
Let  fi  be  a  fundamental  system  of  solutions  of  y  —  Ay.  Let  hi  be  a  special 
solution  of  the  system  y  =  Ay -h  gi{t).  Then  by  the  superposition  principle,  a 
special  solution  of  the  system  S  is  given  by  Note  that  here  n’s  may 

be  regarded  as  constants  or  as  free  parameters.  Then  it  is  straightforward  to 


68 


H.  Anai  and  V.  Weispfenning 


write  down  first-order  formulas  describing  the  respective  forward  and  backward 
reach  sets  and  control  parameter  sets  in  terms  of  evaluations  of  the  basic  func¬ 
tions  the  given  formulas  ip{xi, . . .  ,Xn),  ,  a:^)  and 

a  quantifier-free  formula  ^{yu->  •  •  •  ? 2/in,  •  •  • ,  2/ni)  •  •  • ,  ynn)  describing  the  com¬ 
bined  range  of  (/j,  /ii, ,  hk),  as  a  semialgebraic  set.  All  these  formulas 

will  involve  several  quantifiers  over  real  numbers.  By  real  quantifier  elimination 
one  can  construct  equivalent  quantifier-free  formulas,  and  thus  get  the  desired 
semialgebraic  descriptions. 

We  will  exhibit  concrete  first-order  formulas  for  some  reach  set  problems  and 
control  parameter  set  problem.  The  remaining  cases  are  handled  similarly  in 
[5].  The  forward  discrete  reach  set  problem  can  be  described  by  the  following 
formula  and  hence  be  solved  by  real  quantifier  elimination  and  evaluation  of  the 
basic  functions  at  finitely  many  points. 

3X1...  ^ifi  +  Ei  nhi){to)  A  [a;=j  yj  =  (5:;.  Xifij  +  Tihijitl)) 

V  •  •  •  V  A"=1  Vi  =  (Ei  +  Ei  rihii)it^)]). 

Next  suppose  we  have  a  quantifier-free  formula  //(^/n,  •  •  • , 2/in,  ■  •  • , 2/ni,  •  •  < , 
2/nn,  2:11, ... ,  ,  Zkiy . . . ,  Zkn)  describing  the  combined  range  of  (/i, . . , ,  /„, 

hu  . . . ,  hk)  on  the  interval  [to,  00)  or  [to,  h].  So  fi(yu,. . . ,  Zkn)  holds  for  n{k+n)- 
tuple  in  if  and  only  if  this  tuple  is  in  the  combined  range  of  (/i, . . . ,  /„, 

hi,...,hk)  on  the  given  interval.  Then  the  forward  bounded  and  unbounded 
reach  set  problem,  respectively,  can  be  described  by  the  following  formula  and 
hence  solved  by  real  quantifier  elimination: 

3xi...  3xn[y:>{Y^i  Xifi  -f-  rihi){to)  A  3?/ii . . .  Bynn^zu  ■  ■  •  3zfcn(M(yn,  •  •  • ,  Zkn) 
^  Vj  =  (Ei  ^iVij  +  Ei  nzi^))]. 

With  the  same  formula  /r,  the  backward  bounded  and  unbounded  reach  set 
problem,  respectively,  can  be  described  by  the  following  formula  and  hence  solved 
by  real  quantifier  elimination: 

3xi .  •  =  (Ei^ifij  +  Ei  A  V?/n  . .  .Vy^^Vzn  . . 

Finally,  the  bounded  interval  control  problem  and  the  unbounded  interval 
control  problem,  respectively,  can  be  described  by  the  following  formula  and 
hence  solved  by  real  quantifier  elimination: 

3X1  ..  .  3Xn[ip{Yi  ^ifi  +  Ei  Uhi){to)  A  V2/11  .  .  .  Vynn'^Zii  .  .  .  Wzkn 
(m(2/11,  ■  •  .  ,  Zkn)  ^(Ei  +  Ei  W)]- 

As  a  corollary  to  these  semialgebraic  parametric  descriptions  of  reach  sets 
we  also  obtain  semialgebraic  descriptions  of  the  corresponding  reach  sets,  where 
the  control  parameters  range  over  a  prescribed  semialgebraic  set  C. 

Corollary  1.  Let  C  C  be  a  semialgebraic  sets  of  control  parameters  de- 
scnbed  by  a  quantifier-free  formula  7(7’!, . .  .,rk).  Let  p{yu. . .  ,2/n,n, . . .  ,rA:)  be 
a  quantifier-free  formula  describing  a  forward/backward  reach  set  relative  to  the 
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control  parameters  r.  Then  the  corresponding  reach  set  for  arbitrary  control  pa¬ 
rameter  values  in  C  is  described  by  the  formula  3ri . . .  3rA:(7(r)  A  p{y,r)),  and 
Vri . .  yrk{'y(r)  p{y,L))y  respectively^  and  hence  is  also  a  semialgebraic  set 


Example  1.  Consider  the  inhomogeneous  system  ^  y  =  Ay  -\-b  with 


Then  the  basic  functions  are 


A  quantifier- 


free  formula  2/12?  2/21,2/22?  ^n,  zi2, 221?  ^22)  describing  the  combined  range  of 

these  functions  for  t  €  [0, 00)  is  obtained  as  follows:  Notice  that  the  range  of  e~‘ 
on  [0, 00)  is  exactly  (0,1],  and  that  e*  =  =  l/(e“^)^. 

So  p  can  be  taken  as  the  formula 

0  <  2/11  <  1  A  32/12  =  22/11  A  2/212/11  =  -1  A  2/222/11  =  1  A 
zii=Q  A  3zi22/ii  =  1  a  2z2iyii  =  1  A  Z22  =  0. 


3  Exact  Transcendental  Implicit izat ion 

Here  we  consider  cases,  where  the  unbounded  and  bounded  transcendental  im- 
plicitization  problem  for  given  functions  fi  :  /  — >  (1  ^  ^  an 

exact  solution.  Notice  that  the  transcendental  implicitization  problem  refers 
only  to  the  component  functions  fij{t)  of  /i(i);  the  grouping  of  these  compo¬ 
nent  functions  into  vector- valued  functions  is  irrelevant  here.  So  we  may  as¬ 
sume  w.l.o.g.  that  k  =  l  and  that  we  deal  with  a  single  vector-valued  function 
f(t)  :=  (/i(t), . . . ,  /n(0)'  Then  the  exact  transcendental  implicitization  problem 
is  to  determine  the  range  of  f{t)  on  an  unbounded  interval  [to,  00),  or  a  compact 
interval  [fo,^i]  contained  in  L  Since  the  /  is  continuous,  this  range  is  always  a 
connected  subset  of  R’^. 

In  particular  for  n  =  1  the  range  is  a  real  interval  J;  moreover  J  is  compact 
for  the  bounded  implicitization  case.  In  the  unbounded  implicitization  case  J  is 
compact  iff  /  is  bounded  on  [to,  00),  otherwise  it  is  a  closed  semiinfinite  interval 
or  all  of  R.  In  particular  J  is  always  a  semialgebraic  set  that  can  computed 
explicitly  from  upper  and  lower  bounds  for  /.  In  other  words  the  unbounded 
and  the  bounded  transcendental  implicitization  problem  always  has  a  positive 
solution  for  n  =  1. 

For  n  =  2  there  are  two  well-known  cases,  where  exact  unbounded  and 
bounded  implicitization  is  possible,  namely  the  sin-cos-pair  and  the  sinh-cosh- 
pair:  If  /  has  components  fi  :=  cos{p{x)),  f2  :=  sin(p(x)),  where  p{x)  is  a  real 
polynomial  of  positive  degree,  then  the  range  of  p(x)  on  [to?  00)  includes  an 
unbounded  interval;  consequently  the  range  of  /  on  [to?co)  is  exactly  the  unit 
circle  {(xi,X2)  \  xj  A  xl  =  1}.  On  a  bounded  interval  [to?^i],  the  range  of  p{x) 


^  This  is  taken  from  [10]  (p.586  example  3.13). 
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is  again  a  compact  interval,  and  so  the  range  of  /  is  a  connected  subset  of  the 
circle  that  can  be  easily  computed  as  semialgebraic  set  from  the  range  of  p{x). 
For  the  hyperbolic  ca^e,  where  /i  :=  cosh(p(a:)), /s  ;=  sinh(p(a:)),  the  situa¬ 
tion  is  analogous,  except  that  the  role  of  the  circle  is  replaces  by  the  hyperbola 
{{xi,X2)  \x\~xl^  1}. 

The  next  theorem  shows  that  exact  transcendental  implicitization  is  pre¬ 
served  under  composition  of  functions  in  a  very  general  sense: 

Theorem  1.  Let  f(t)  :=  ,  fki^))  be  a  vector  valued  function  such  that 

the  range  of  f  on  every  compact  or  unbounded  closed  interval  I  is  a  semialge¬ 
braic  set  described  by  a  quantifier-free  formula  (pi{xi, . . .  ,Xk).  Let  g  be  a  con¬ 
tinuous  real  function  defined  on  some  compact  or  upper  semiinfinite  closed  in¬ 
terval  I .  Let  hi  (1  <  i  <  n)  be  semialgebraic  real  functions  defined  on  some 
subset  of  extending  the  range  of  f.  Let  . . . ,  t/)  he  quantifier-free 

formulas  defining  the  graph  {(xi, . . . , 2/)  |  y  =  hi{xi, . . .  ,Xn)}  of  hi.  Then 
the  vector-valued  function  f*{t)  :=  {fi{t),...,f*{t))  with  components  ff{t) 
b'iifiigit)), . .  .,fn{g{t)))  fori  <i  <n  has  a  semialgebraic  range  described  by  the 
formula  'ip{xi, ...  ^Xn)  :=  Byi . . .  A  /\^=i  Pi{yi, . . .  ,yn,Xi)), 

where  J  is  the  range  of  g{t)  on  I'. 

The  proof  is  obvious.  Notice  that  the  algorithmic  quantifier  elimination  for  the 
ordered  field  of  real  numbers  this  formula  is  required  in  order  to  transform  the 
formula  -0  into  an  equivalent  quantifier-free  formula  that  describes  the  range  of 
f*  as  a  semialgebraic  set.  Typical  instances  of  g  and  hi  are  real  polynomials  or 
real  rational  functions.  The  method  can  in  particular  be  applied  to  the  situation, 
where  /  consists  of  a  sin-cos-pair  or  a  sinh- cosh- pair  as  described  above.  Other 
interesting  examples  are  pairs  {p,p'),  where  p{t)  is  a  Weierstrass  p-function 
[1].  Then  the  range  of  (p,  p'fi  on  a  large  enough  interval  is  a  real  elliptic  curve 
{(^?  y)  \  =  ^x^  —  g2X  —  ^3}.  See  [5]  for  the  more  examples. 


4  Semialgebraic  Implicitization  for  Simple  Elementary 
Functions 


In  this  section  we  characterize  those  cases  of  linear  differential  systems  S  with 
constant  coefficients  and  “simple  right  hand  side”,  where  an  exact  implicitiza¬ 
tion  of  the  system  of  basic  functions  for  S  is  possible.  The  condition  on  the  right 
hand  side  b{t)  of  the  system  is  as  follows:  All  components  hi  (t)  of  b{t)  are  R-linear 
combinations  of  functions  of  the  form  cos{u;it),  t^^ e^^Hm{u;it) ,  where  di 

are  non-negative  integers  and  ai,u)i,ai  are  real  numbers.  Then  it  is  well  known 
that  a  special  solution  of  the  inhomogeneous  system  and  the  fundamental  solu¬ 
tions  of  the  homogeneous  system  are  again  real  linear  combinations  of  functions 


of  this  kind.  We  call  linear  systems  of  this  form  regular  and  functions  of  type 
t  e  cos{ujt),  sin[Lijt),  with  a,  a;,Q;  real  numbers  simple  elementary  func¬ 
tions.  In  some  special  cases  of  regular  systems,  it  has  been  shown  how  to  solve 
the  reach  set  problem  by  an  implicit  semialgebraic  implicitization  of  functions 
of  the  following  type  in  [23,24,22]  :  (i)  real  polynomials  pi(t),  (ii)  exponential 
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functions  with  rational  values  of  a*,  (iii)  trigonometric  functions  cos(a;it), 
sm{uJit),  for  rational  uji. 

In  the  following  we  show  that  for  simple  elementary  functions  there  are  only 
few  more  cases  which  allow  unbounded  exact  semialgebraic  implicitization;  all 
these  cases  are  covered  by  Theorem  1  of  the  last  section.  In  most  of  the  re¬ 
maining  cases  the  exact  semialgebraic  implicitization  problem  is  unsolvable.  In 
fact  we  provide  a  complete  characterization  of  those  cases,  where  unbounded 
semialgebraic  implicitization  is  possible. 

Let  f{t)  :=  (/i(t), . .  • ,  fn(t))  with  non-constant,  pairwise  different  component 
functions  fi{t)  :=  cos(a;it),  or  fi{t)  :=  sm{u;it),  where  di  are  non¬ 

negative  integers  and  ai^uJi  are  real  numbers.  Moreover  we  assume  that  the 
functions  fi  appear  in  cos-sin-pairs,  whenever  Ui  ^0. 

Theorem  2.  Let  f  :  [to,  oo)  — >  be  as  above  and  let  n  >  2.  Then  the  range 

of  f  is  a  semialgebraic  set  iff  one  of  the  following  holds: 

1.  For  all  1  <  i  <  n,  fi{t)  := 

2.  For  alll  <i  <n,  di  =  0,  fi{t)  :=  and  dimQ{span{ai, . . . ,  On))  <  1- 

5.  For  all  1  <  i  <  n,  di  ^  0,  ^  0,  fi(t)  :=  and 

diTnQ{span{ai, . . .,««))  <  1,  o,rid 

4.  For  alll<i<n,  fi{t)  :=  cos{u)it),  or  fi{t)  :=  sin(u;it),  and 
dim(Q{span{ui, . . .  ,a^n))  <  T 

Moreover  in  these  positive  cases  a  quantifier-free  formula  describing  the  range 
of  f  can  be  computed  algorithmically  over  the  reals. 

Idea  of  the  Proof.  In  the  cases  mentioned  above  the  unbounded  semialgebraic 
implicitization  is  always  achieved  by  the  methods  of  the  previous  section,  in 
particular  Theorem  1.  It  remains  to  show  that  in  all  other  cases  the  range  of  / 
is  not  a  semialgebraic  set.  This  requires  a  case  distinction.  In  each  case  we  show 
that  the  assumption  that  the  range  of  /  is  semialgebraic  leads  to  a  contradiction. 
Based  on  the  assumption  that  the  range  of  /  is  semialgebraic  we  construct  new 
semialgebraic  sets  with  impossible  properties.  Either  this  set  is  one  dimensional 
such  that  neither  the  set  nor  its  complement  is  a  finite  union  of  intervals  or  it 
describes  the  graph  of  a  semialgebraic  function  with  an  impossible  rate  of  growth 
(compare  [9]).  See  [5]  for  details  of  the  proof. 

This  theorem  clearly  shows  the  limitations  of  the  approach  presented  in  [23, 
24].  In  fact  we  have  the  following  immediate  corollary: 

Corollary  2.  Let  y  =  Ay  with  constant  n  x  n-matrix  A  be  a  homogeneous 
system  of  linear  differential  equations.  Then  exact  semialgebraic  implicitization 
is  possible  for  a  fundamental  system  of  solutions  of  the  system  iff  one  of  the 
following  cases  holds: 

1.  All  eigenvalues  of  A  are  zero,  i.e.  A  is  a  nilpotent  matrix. 

2.  All  eigenvalues  Ai,...,An  of  A  are  non-zero,  pairwise  distinct  reals,  and 
dimQ{span{Xi, . . . ,  A^))  <  1. 

3.  All  eigenvalues  Ai, . . . ,  An  of  A  are  purely  imaginary,  say  of  the  form  \i  = 

with  non-zero  pairwise  distinct  reals  pi,  and  dimQ{span{pi, . . . ,  pn))  ^  1* 
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5  Approximate  Solutions 

In  this  section  we  study  the  cases,  where  an  exact  semialgebraic  unbounded  or 
bounded  implicitization  is  definitely  not  possible.  In  these  cases  we  want  to  find 
a  semialgebraic  superset  of  the  true  forward  reach  set  and  a  semialgebraic  subset 
of  the  true  backward  reach  set  or  the  true  control  parameter  set,  both  if  possible 
such  that  the  set  difference  to  the  true  reach  set  or  control  parameter  set  is  in 
some  sense  small  enough.”  Then  an  inspection  of  the  reduction  formulas  shows 
that  an  overestimation  of  the  implicitization  problem  leads  to  an  overestimation 
of  the  forward  reach  set  and  an  underestimation  of  the  backward  reach  set  and  of 
the  control  parameter  set  i.e.  for  “safe”  estimations.  Hence  we  are  reduced  to  the 
problem  of  finding  a  semialgebraic  superset  of  the  true  range  of  a  transcendental 
vector  valued  function  on  a  compact  or  upper  semiinfinite  closed  interval. 

One  strategy  to  find  overestimations  of  the  range  is  separation  of  variables: 
It  comes  in  two  flavours:  Separation  of  variables  in  different  components,  and 
separation  of  variables  in  products. 

Separation  of  variables  in  different  components  :  Let  f{t)  = 

be  defined  on  an  interval  I.  Then  separation  of  variables  in  dif¬ 
ferent  components  yields  the  function  g{t)  =  (/i(ti), . . . ,  /„(tn))  defined  on  the 
cube  1“^  with  range(5)  D  range(/).  The  range  of  g  is  easily  computed  as  a  box 
Ji  X  •  • .  X  J„,  where  Ji  is  the  range  of  fi.  Notice  that  this  box  is  in  fact  the 
smallest  box  containing  the  range  of  /. 

Separation  of  variables  in  products  :  Suppose  the  component  functions 
of  the  given  functions  are  products  fi{t)  :=  fi^iit)  •  •  ‘  fi^rn{t)^  where  each 
is  defined  on  the  interval  /.  Put  g^{t)  (/i,„ . . . ,  Then  each  g^  is  ’also 

defined  on  the  interval  /.  Let  Bj  be  the  range  gj,  and  put  C  \=  Hi 
where  the  multiplication  is  performed  on  the  elements  componentwise.  Then  C 
is  obviously  a  superset  of  the  range  of  /. 

Example  2.  Let  I  be  the  upper  semiinfinite  interval  [0,  oo). 

1.  Let  /i  :=  cos(t),  /2  :=  sin(t).  Then  the  true  range  of  /  is  the  unit  circle. 
Separation  of  variables  in  different  components  yields  as  overestimation  the 
closed  unit  square. 

2.  Let  fi  :=  cosh(t),  /2  :=  sinh(t).  Then  the  true  range  of  /  is  the  hyperbola 

{{x,y)  \  =  1}.  Separation  of  variables  in  different  components  yields 

as  overestimation  the  “quadrant”  \  x,y  >  1}. 

3.  Let  fi  :=  e*cos{t),  /2  ;=  e^sin(t).  Then  the  true  range  of  /  is  an  expanding 

exponential  spiral.  Separation  of  variables  in  different  components  yields  as 
overestimation  the  full  plane  R^.  Separation  of  variables  in  products  yields 
as  better  overestimation  the  annulus  {(x,y)  |  >  ij., 

4.  Let  /i  :=  e  ^  cos(t),  /2  :=  e~^  sin(t).  Then  the  true  range  of  /  is  a  contracting 
exponential  spiral.  Separation  of  variables  in  different  components  yields  as 
overestimation  a  closed  box  {{x,y)  \  -e'^  <  x  <  1,-e^^/^  <  y  < 
Separation  of  variables  in  products  yields  as  overestimation  the  closed  disk 
{(x,y)  I  x'^  -\-  y^  <  1}.  These  approximations  are  incomparable.  So  their 
intersection  is  a  common  improvement  of  both. 
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6  Complexity 

In  this  section  we  briefly  discuss  the  complexity  of  our  algorithms.  From  the 
results  on  complexity  of  quantifier  elimination  in  [7]  we  can  give  upper  bounds 
for  the  asymptotic  complexity  of  our  approach: 

Discrete  point  reach  set  problems  are  described  by  purely  existential  formu¬ 
las.  Hence  the  complexity  of  quantifier  elimination  is  at  most  simply  exponential 
in  the  dimension  of  the  differential  system.  For  fixed  dimension  it  the  compu¬ 
tation  runs  in  a  polynomial  time.  The  complexity  of  bounded  and  unbounded 
reach  set  problems  is  the  same  as  for  the  discrete  reach  set  problem  for  a  fixed 
number  m  of  points.  The  backward  discrete  reach  set  problems  can  be  solved  in 
singly  exponential  time.  The  complexity  of  backward  bounded  and  unbounded 
reach  set  computation  is  of  type  (generalized  singly  exponential).  The  up¬ 

per  complexity  bounds  for  the  control  parameter  set  problems  are  same  as  for 
the  corresponding  backward  reach  set  problems. 


7  Computational  Example  in  redlog  and  qepcad 


In  this  section  we  report  on  experimental  results  in  reach  set  and  control  param¬ 
eter  set  computation.  In  [5]  we  have  presented  experimental  results  for  numerous 
examples  that  illustrate  the  different  problem  types  and  solution  methods.  Here 
we  display  only  one  of  these  examples  with  non-constant  coefificients  to  show 
the  generality  of  the  approach.  All  computations  are  performed  in  the  REDLOG 
package  [14]  of  REDUCE  3.7  and  QEPCAD  [12]  ^  .  The  main  algorithm  employed  is 
the  linear  and  quadratic  quantifier  elimination  [25,31]  of  REDLOG  and  quantifier 
elimination  based  on  cylindrical  algebraic  decomposition  [12]  of  QEPCAD. 


Example  3.  Consider  the  inhomogeneous  system  y  =  Ay  +  h  with 


A:: 


/  0  2t\  ,  /2tcos(t^)\ 

(-2i  oj’  ^^=’'A2tsin(t^)j 


Then  basic  functions  are  (  ^ system  we 

illustrate  the  computations  in  the  forward/backward  unbounded  reach  set  and 
the  control  parameter  set  problems  below  (Note  that  we  set  to  ~  0): 


•  Forward  unbounded  reach  set:  A  quantifier-free  formula  /i(2/ii,  yi2, 2/21,2/22, 
>2^11,^12)  is  obtained  from  the  following  first-order  formula  po 


Po  =  3u3v{u‘^-\-v‘^  =  lAyii  =  vAyi2  =  uAy2i  =  uAy22  =  -vAzu  =  vAzu  =  0)) 


by  using  quantifier  elimination.  By  using  REDLOG  we  have 

/^  •=  ?/ii  +  2/12  "  1  0  ^  2/11  +  2/22  =  0  A  yn  ~  zn  =  0  A  y^  -  1/21  =  0  A  Z12  =  0 


^  All  the  computations  are  executed  on  a  SUN  SPARC  station  Ultra  I  (140MHz). 
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in  10  ms.  Then  we  set  ri  =  1  and  moreover  =  (0  <  <  1  A  0:2  =  0). 

Then  forward  unbounded  reach  set  problem  is  solved  by  using  real  quantifier 
elimination  for  the  following  first-order  formula  f  reach] 

f reach  =  ^xi{(f  A  freachaux) 

where 

freachaux  =  3yiiByi23y2i3y22^zii3zi2{fi  Ayi  =  ^lyn  +  ^2^21  +  nzu 

Ay2  =  xiyi2  +  X2y22  +  nzu) 

By  using  QEPCAD  for  f  reach  we  obtain  as  an  answer  for  the  forward  unbounded 
reach  set;  yj  +  42/2  -  4  <=  0  in  10  ms. 

•  Backward  unbounded  reach  set:  /x  is  the  same  formula  as  in  forward 

unbounded  reach  set.  We  also  set  ri  =  1  and  'ip{xi,X2)  =  (-|  <  ^  A  < 

<  |)-  Then  the  backward  unbounded  reach  set  problem  is  solved  by  using 
real  quantifier  elimination  for  the  following  first-order  formula  breach] 

breach  =  3xi3x2{yi  =  X2  A  y2  —  Xi  A  breachaux) 

where 

6reac/iaua:  =  V2/iiV2/i2V2/2iV2/22V>2:iiV2:i2(/x  {-|  <  xiyix  +  X2y2i  +  riz^  <  \ 

A  -  i  <  xiyi2  +  X2y22  +  rizi2  <  \) 

By  using  REDLOG  for  breach  we  obtain  in  420  ms  a  semialgebraic  description  of 
the  backward  unbounded  reach  set  consisting  of  21  atomic  formulas. 

•  Control  parameter  set:  The  formula  /x  is  the  same  as  in  the  reach  set  cases. 
We  also  set  V?  =  (0  <  ari  <  1  A  0:2  =  0)  and  ip{xi,X2)  =  (-i  <  a^i  <  |  A  -^  < 

<  I).  Then  control  parameter  set  problem  is  solved  by  using  real  quantifier 
elimination  for  the  following  first-order  formula  pcontrol] 

control  =  3xi  {ip  A  controlaux) 

where 

controlaux  =  V2/iiV2/i2V2/2iV2/22V2:iiVzi2(/x  (— ^  <  xix/n  +2:22/21  +^1^11  <  \ 

A  -  i  <  xiyi2  +  2:22/22  +  rizi2  <  \) 

By  using  REDLOG  for  control  we  obtain  in  70  ms  a  semialgebraic  description  of 
control  parameter  set  consisting  of  12  atomic  formulas.  It  can  be  simplified  to 
the  result  -1  <  ri  <  ^  by  hand  calculation. 


8  Conclusions 

In  this  paper  we  have  studied  forward  and  backward  reach  set  and  control  pa¬ 
rameter  set  problems  for  continuous  parametric  open-loop  systems  described  by 
a  system  of  parametric  linear  differential  equations  with  arbitrary  coefficients. 

The  approach  using  quantifier  elimination  was  introduced  into  reach  set  com¬ 
putations  in  [29].  We  extend  their  ad  hoc  approach  for  special  types  of  differential 
systems  to  a  systematic  study  of  the  type  of  results  obtainable  by  an  approach 
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via  real  quantifier  elimination.  Thus  we  obtain  a  much  wider  systematic  frame¬ 
work  applicable  to  a  considerably  larger  class  of  systems.  The  main  observation 
is  that  all  the  problems  can  be  reduced  by  exact  symbolic  algorithms  to  an  im- 
plicitization  problem  for  certain  basic  transcendental  functions  associated  with 
the  given  system. 

We  have  proved  a  theorem  that  determines  the  exact  classes  of  vector- valued 
functions  of  the  kind  arising  in  linear  differential  systems  with  constant  coeffi¬ 
cients,  where  exact  semialgebraic  implicitization  is  possible.  As  a  corollary  we 
have  obtained  the  exact  limitations  of  the  approach  of  [23,24]  for  linear  differ¬ 
ential  systems  with  constant  coefficients  and  simple  elementary  inhomogeneous 
part.  We  have  also  proposed  several  ways  to  overcome  these  limitations  by  ap¬ 
proximate  computations.  The  problems  have  been  illustrated  by  examples  com¬ 
puted  in  the  REDLOG-package  of  REDUCE  and  QEPCAD. 

Further  research  will  be  concerned  with  an  extension  of  these  results  to  hybrid 
systems. 
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Abstract.  In  this  work  we  present  a  novel  control  design  methodology 
for  under- actuated  mechanical  systems.  As  part  of  the  design  process  we 
use  the  reachability  analysis  tool  d/dt  [ABDM99,D00]  to  see  whether 
there  is  a  switching  sequence  which  can  drive  the  system  to  a  desired 
periodic  orbit.  Much  of  the  work  in  the  design  of  the  control  law  is  done 
manually  using  classical  control  techniques  (unlike  the  fully-automatic 
approach  advocated  in  [ABD'''00]),  and  d/dt  is  used  to  complement 
these  techniques.  We  hope  this  work  will  contribute  to  the  proliferation 
of  reachability-based  techniques  to  the  control  engineer’s  tool  box. 


1  Introduction 

The  algorithmic  approach  to  the  analysis  of  hybrid  systems,  first  put  forward 
explicitly  in  [ACH+QS],  is  inspired  by  a  computer  science  approach  to  verifica¬ 
tion  of  automata.  The  system  under  consideration  is  viewed  as  a  generator  of 
trajectories  and  the  problem  of  verification  consists  of  checking  whether  there 
is  an  individual  trajectory  which  violates  some  specification,  e.g.  reaches  a  bad 
state.  Likewise,  the  controller  synthesis  problem  is  phrased  as  restricting  sys¬ 
tematically  the  set  of  all  possible  behaviors  in  order  to  satisfy  a  property.  The 
algorithmic  approach  consists  in  making  a  brute-force  search  in  the  state-space, 
based  only  on  the  description  of  the  system  dynamics.  Initially  this  approach 
has  been  applied  to  restricted  classes  of  hybrid  systems  where  the  continuous 
dynamics  has  a  constant  derivative  in  every  state,  see  e.g.  [AD94]  for  timed 
automata,  and  [ACH+95,AMP95,HHW97]  for  hybrid  automata.  More  recently 
attempts  have  been  made  to  lift  this  approach  to  systems  with  non-trivial  dy¬ 
namics.  In  particular,  some  of  the  authors  were  involved  in  the  development  of 
d/dt,  a  tool  for  verification  and  controller  synthesis  for  hybrid  systems  with  lin¬ 
ear  continuous  dynamics  [ABDM99,D00].  The  synthesis  algorithm  implemented 

*  This  work  was  partially  supported  by  the  European  Community  Esprit- LTR  Project 
26270  VHS  (Verification  of  Hybrid  systems)  and  the  Prench-Israeli  collaboration 
project  970MAEFUT5  (Hybrid  Models  of  Industrial  Plants). 

M.D.  Di  Benedetto,  A.  Sangiovanni-Vincentelli  (Eds.);  HSCC  2001,  LNCS  2034,  pp.  77-88,  2001. 
Springer- Verlag  Berlin  Heidelberg  2001 
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in  d/dt  [ABD+00,D00]  suggested  a  very  idealistic  scenario  for  switching-based 
control:  the  user  defines  the  dynamics  at  the  various  modes,  as  well  as  the  con¬ 
trol  objective,  and  the  tool  automatically  generates  the  appropriate  conditions 
for  mode  switching. 

This  approach  attempts  to  obtain  the  general-purpose  flavor  of  discrete  veri¬ 
fication  tools  and  it  is  still  very  remote  from  control  engineering  practice.  In  the 
continuous  world,  every  class  of  systems  has  its  own  special  character  as  well  as 
its  corresponding  mathematical  tricks  which  are  used  extensively  by  engineers 
during  the  controller  design  process.  Coordinate  transformations,  dimensional¬ 
ity  reduction,  simplifying  assumptions  or  linearization  cannot  be  captured  by 
straightforward  reachability  analysis. 

In  this  paper  we  show  how  reachability-based  techniques  can  be  combined 
with  more  “knowledge-based”  methods  in  order  to  derive  control  strategies  for  a 
non-trivial  class  of  dynamical  systems,  namely  under- actuated  mechanical  sys¬ 
tems.  We  propose  a  general  methodology  for  designing  controllers  for  such  sys¬ 
tems  and  demonstrate  it  on  a  double-pendulum  example.  The  complexity  of  the 
system  as  given  initially  exceeds  the  current  capabilities  of  reachability- based 
tools:  its  dynamics  is  non-linear  and  control  is  done  using  continuous  actuation. 
Moreover,  the  system  is  of  dimension  n  while  the  dimensionality  of  the  available 
control  is  m  <  n.  The  proposed  approach  to  control  this  system  by  switching  is 
based  on  the  following  principles. 

1.  The  state-space  can  be  transformed  and  partitioned  via  a  diffeomorphism  (j? 
into  an  m-dimensional  part  ei  and  an  (n  —  m)-dimensional  part  62. 

2.  Using  standard  control  techniques,  ei  can  be  controlled  to  zero.  Given  this 
control,  the  remaining  part  is  a  closed  system  which  defines  the  dynamics  of 
62  (called  the  Zero  dynamics). 

3.  Each  diffeomorphism  induces  a  different  control  law  for  its  zero  dynamics 
and  hence  a  particular  “mode”  for  the  dynamics  of  the  the  uncontrolled 
part  of  the  system.  We  use  a  parameterized  family  of  diffeomorphisms  which 
becomes  finite  after  discretizing  the  parameters. 

4.  The  dynamics  of  62  at  each  mode  can  be  linearized  around  its  equilibrium 
point.  It  is  possible  to  choose  the  parameters  so  that  the  linearized  system 
has  periodic  orbits  in  every  mode.  It  should  be  kept  in  mind  that  the  validity 
of  the  linear  model  is  restricted  to  the  neighborhood  of  the  equilibrium. 

5.  If  our  goal  is  to  reach  a  specific  periodic  orbit,  we  can  achieve  it  by  a  sequence 
of  mode  switchings.  At  each  mode,  however,  a  different  quantity  is  controlled 
to  zero.  Hence,  when  we  switch  from  controlling  ei  to  controlling  the 
latter  should  already  be  close  to  zero.  This  restricts  the  parts  of  the  state- 
space  of  the  62  system  where  switching  is  allowed  and  leads  to  modeling 
the  system  as  a  hybrid  automaton  where  the  transition  guards  reflect  these 
constraints. 

The  role  of  d/dt  is  then  to  check  whether,  based  on  the  hybrid  automaton 
representation,  it  is  possible  to  reach  from  one  orbit  to  another  by  mode  switching 
and  how  much  time  it  takes. 
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2  Control  of  Under- Actuated  Mechanical  Systems 


2.1  Under- Actuated  Mechanical  Systems 

We  consider  the  class  of  jointed  mechanical  systems  without  flexibilities,  the 
dynamics  of  which  is  given  by  Lagrange  equations: 

M{q)q  +  N{q,q)  =  Wr  (1) 


where  M  is  the  symmetric  positive  definite  matrix  defining  the  kinetic  en¬ 
ergy  and  N  gathers  generalized  gravity,  Coriolis  and  centrifugal  forces;  q  is  the 
n- dimensional  vector  of  generalized  (joint)  coordinates;  F  includes  all  external 
generalized  forces  and  W  is  a  constant  matrix. 

If  we  now  assume  that  the  generalized  forces  are  only  actuation  torques/forces 
(i.e  the  system  is  friction-free  and  no  other  potential-based  actions  occur),  then 
the  system  is  called  under- actuated  if  rank  (W)  <  n.  Without  loss  of  generality. 


we  can  consider  that  W  — 


with  m  <  n  the  number  of  actuators. 


2.2  Zero  Dynamics 


Let  us  consider  a  diffeomorphism 


q  <t>(q)  = 


(2) 


where  ei  is  m-dimensional.  Then,  the  dynamics  (1)  projected  on  the  constraint 
ei  =  0  is  called  the  zero  dynamics  associated  with  (p.  It  is  given  by: 


P{q)(M{q)q^N{q,q))  =  0  (3) 

with  P  =  JiM~^  the  projection  operator,  in  which 

A  control  objective  can  therefore  be  to  bring  the  system  to  this  zero 
dynamics,  specified  by  the  goal  task  ei  =  0,  and  to  stabilize  it.  Since  dim  (ei)  = 
dim  (r),  all  the  available  actuation  forces/torques  have  to  be  used  for  that  pur¬ 
pose.  In  fact,  that  can  be  done  trough  partial  decoupling/feedback  linearization: 
it  can  be  easily  seen  that  using  the  control 

r  =  {JiM-^W)-\u  -  M  +  JiM-^N)  (4) 

we  obtain  e'l  =  u,  assumed  that  JiM~^W  is  nonsingular.  It  then  remains  to 
specify  an  adequate  input  u  which  stabilizes  ei,  asymptotically  or  in  finite  time, 
in  order  to  drive  the  system  to  the  zero  dynamics.  Once  reached,  its  motion  is 
then  governed  by  eq.  (3),  which  is  free,  since  no  more  control  is  available.  In 
many  cases,  this  free  motion  is  a  periodic  orbit.  The  idea  now  is  to  specify  such 
a  periodic  orbit  as  a  final  goal,  recalling  that  we  can  consider  the  choice  of  (p 
as  a  way  to  modify  it.  The  problem  addressed  in  the  following  is  then  to  study 
the  reachability  of  this  behavior  starting  from  given  initial  conditions,  using  a 
sequence  (/)i, (^2  •  •  •  ?  i-e  successive  jumps  from  an  orbit  to  another  one. 
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2.3  Handling  the  Periodic  Orbits 

Let  us  consider  the  case  where  m  =  n  — 1,  i.e.  the  zero  dynamics  can  be  expressed 
using  a  single  coordinate  denoted  by  xj .  When  the  phase  portrait  of  the  system 
is  a  closed  curve  O,  this  periodic  orbit,  which  characterizes  the  zero  dynamics, 
can  be  uniquely  specified  by  a  pair  where  is  a  point  on  the  orbit,  for 

example  the  initial  conditions.  Let  us  assume  (assumption  AO)  that  the  equation 
of  O  in  the  phase  plane  is  of  the  form  V{xi,xi)  ~V  =  0,  the  invariance  being 
expressed  by  V  =  0.  y  is  a  so-called  Lyapunov  function.  For  a  non-actuated 
conservative  mechanical  system,  the  natural  V  is  the  mechanical  energy.  Since 
it  is  not  the  case  here,  V  can  only  be  called  by  analogy  the  “energy”  level  of  the 
orbit. 

Let  us  now  consider  the  particular  case  where  the  set  of  consists  of  func¬ 
tions  of  given  analytical  form  depending  on  a  fc-dimensional  vector  of  real  param¬ 
eters  p.  Then  p  can  be  considered  as  an  auxiliary  control  of  the  system.  Giving 
some  bounds  to  the  parameters  and  the  variables,  so  that  they  range  over  Dp 
and  Dxo ,  respectively,  the  set  of  all  possible  orbits  for  the  system  is 

0  =  {0{p,X’>)-.peDpX°GDxo}. 

When  V  is  known,  the  set  can  also  be  parameterized  by  p  and  V. 

The  problem  we  address  now  is  the  following:  let  us  define  a  desired  behavior 
of  the  system  as  a  goal  orbit  O*  ;  then,  given  an  initial  orbit  Oq  ^  O*,  can  we 
reach  O*  by  modifying  p?  We  don’t  consider  here  related  problems  of  automatic 
control:  existence  of  the  orbits,  active  stabilization,  continuous  control  ofp,  which 
will  be  addressed  in  forthcoming  papers.  Instead,  we  focus  our  attention  on 
a  discrete  approach,  i.e.  to  the  questions:  is  there  a  sequence  of  intersecting 
orbits  allowing  to  reach  O*  through  jumps  on  the  parameters  and  how  long 
time  will  it  take?  Assuming  here  that  these  jumps  are  instantaneous  and  don’t 
disturb  the  overall  behavior  (assumption  Al),  we  can  therefore  forget  the  effect 
of  the  control  (4)  and  consider  for  the  analysis  the  related  set  of  zero  dynamics 
uniquely.  We  are  therefore  led  back  to  a  problem  of  reachability  analysis  of 
a  hybrid  system:  each  discrete  state  is  an  homogeneous  differential  equation 
associated  with  given  values  of  the  parameters;  transitions  are  allowed  when 
orbits  of  different  modes  are  compatible  with  each  other,  i.e.  when  continuous 
state  variables  reach  some  particular  values.  We  will  illustrate  the  approach  on 
the  double  pendulum  example. 

3  The  Case  of  the  Double  Pendulum 

The  considered  testbed  is  the  double  pendulum  depicted  in  Figure  1 .  The  reader 
is  referred  to  [EGP99]  for  details  on  experimental  issues.  Terms  in  eq.  (1)  write 
for  this  system  as: 


M  = 


mil  mi2 

^12  ^22 


(5) 


On  Hybrid  Control  of  Under- Actuated  Mechanical  Systems 


81 


and: 

]\f  ^  f  \  ^  f  Cn  012^  f  Qi\  ,  f 

\N2{Qi^<l2,QiyQ2)  J  \(^2i  C22  J  J  \^2/ 

with: 

TTLii  —  d“  ^2(^2  2Z/1/2C2) 

?^12  ==  ^2(^2  +  L1I2C2) 

71122  ~  ^2^2 

=  —m2Lil2s2q2 
C12  =  -m2Lil2s2{qi  +92) 

<^21  =  m2Lil2s2qi 
C22  =  0 

Gi  =  g{{mili  +  m2Li)sl  +  7712/2512) 

G2  —  gm2l2sl2 


(6) 


(7) 


where  si  :=  sin{qi)  ,  ci  :=  cos{qi)  ,  sij  sin{qi  +  We  consider  the  case 
where  only  the  hip  is  actuated.  Therefore  W  =  Let  us  now  choose  the 

diffeomorphism  (f)  and  the  control  F  such  that 


ei=qi-aq2-b  =  0  £2  =  72  (8) 

where  a  and  b  are  two  real  parameters^.  Therefore  the  zero  dynamics  we  have 
to  consider  is  simply: 


(  {77122  “b  ^77112)72  +  {<^^21  +  ^22)72  +  G2  =  0 
1  7i  =  a72  +  6 


(9) 


where  it  assumed  that  77i22+®’^i2  7^  9  (assumption  A2,  satisfied  when  < 

).  This  system  can  be  expressed  in  the  single  coordinate  q2.  It  is  a 


a  < 


Li-h 


second  order  nonlinear  differential  equation,  for  which  the  natural  state  vector 
isX  =  “  (^^7  ^  ^^^^Lability  analysis,  we  have  to 

linearize  the  system.  Its  equilibrium  points  X*  =  are  solutions  of  <^2(72)  — 


0,  i.e,  for  a  ^  —1  (assumption  A3): 


72  — 


6  -h  A:7r 

1  H"  O' 


(10) 


We  consider  in  the  following  only  the  case  A;  =  0.  The  equation  of  the  system 
linearized  around  the  center  q^  is: 


X  =  Ax 


{-A) 


(11) 


^  Note  that  expression  (8)  specifies  the  desired  spatial  trajectory  of  the  tip  of  the 
double  pendulum,  while  the  “energy”  level  will  set  the  amplitude  and  the  time 
profile  of  its  motion  along  this  trajectory 
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Fig.  1.  A  double  pendulum. 


where  a  =  I2  +  j~Licos{j^).  For  ensuring  the  existence  of  periodic  orbits,  the 
eigenvalues  of  A  have  to  be  imaginary,  which  implies  that  a  has  to  be  strictly 
positive  (assumption  A4).  The  Lyapunov  function  associated  with  the  system, 
i.e  the  energy  level  of  an  orbit  is 

V  =  ^{axl+xl)  (12) 

For  the  purpose  of  reachability  analysis  it  is  more  comfortable  to  work  with 
the  same  system  of  coordinates  in  every  state,  hence  we  transform  the  linear 
dynamics  of  equation  (11)  into  an  affine  dynamics  over  y  =  {q2Tq2)’ 

Finally  we  have  to  remember  that  the  system  is  submitted  to  physical  bounds  on 
the  joints:  qi  G  [qT^"^ ^  Introducing  them  in  (8)  leads  to  linear  constraints 

on  the  parameters. 

When  we  switch  from  (f)  to  </>'  there  might  be  a  transient  period  until  the 
system  settles  in  the  new  zero  dynamics.  In  order  to  make  assumption  Al  (tran¬ 
sitions  are  immediate)  realistic  we  need  to  make  sure  that  e\  and  be  already 
close  to  their  zero.  For  qi  this  means 


e'll  =  -a'q'2-b'\  <  ei 


(14) 
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Since  qi  =  aq2  +  b  this  reduces  to 

\{a-a')q2  +  (b-b')\  <ei  (15) 

For  q2  we  need: 

|e'i|  =  |(a-a')92|  <f2  (16) 

These  conditions,  which  form  rectangles  in  the  phase-space  of  the  zero  dynamics, 
will  be  used  as  transition  guards  in  the  hybrid  automaton  model.  Note  that  these 
conditions  are  symmetric,  i.e.  they  are  the  same,  in  terms  of  q2  and  q2  for  the 
transitions  from  (a',  6')  to  (a,  6).  Of  course,  their  global  physical  interpretation 
does  depend  on  the  source  state  of  the  transition. 

The  system  is  modeled  as  a  hybrid  automaton  with  7  states,  each  representing 
a  pair  (a,  h)  of  parameters  (Figure  2).  At  each  state  the  dynamics  is  of  the  form 
X  =  Ax  +  u  where  A  and  u  for  the  various  states  are: 


Sq  Sj  S2  S3  34  S5  SQ 


01  01  01  01  01  01  01 
-0.0479  0  -0.0878  0  -0.1167  0  -0.1982  0  -0.2326  0  -0.3143  0  -0.3555  0 

0  0  0  0  0  0  0 
0.0011  0.0000  -0.0012  0.0000  -0.0039  -0.0090  -0.0140 


The  transition  guards  are  computed  according  to  (15)  and  (16)  with  ei  =  0.05 
and  €2  =  0-02.  In  addition,  we  restrict  the  transitions  to  happen  between  pairs 
of  “close”  states,  i.e.  \a  —  a'\  <  0.15  and  |6  —  b'\  <  0.1. 


[-3.75,  -1.25]x 


[-3.333,  0.0]  X 


Fig.  2.  The  hybrid  automaton  for  the  double  pendulum.  The  transition  guards  between 
pairs  of  states  are  written  as  products  of  intervals. 
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In  order  to  facilitate  the  experimentation  with  d/dt  we  have  augmented  the 
input  syntax  to  include  parameters  and  formulae  referring  to  them.  For  example, 
state  So  and  its  outgoing  transition  is  specified  as: 

state:  0; 
matrixA : 

0.0  1.0, 

[-12-(a0/ (l+a0))*Ll*cos(b0/(l+a0))]  0.0; 

input:  type  convex_vert 

0.0  [(bO/(l+aO))*(-12-(aO/(l+aO))*Ll*cos(bO/(l+aO)))] ; 
transition: 

label  goOl : 

if  in  guard:  type  rectangle 

[“(-epsl+(b0-bl))/(a0-al)3  [-(epsl+(b0“bl) )/(a0-al)] , 

Ceps2/(a0-al)]  C-eps2/(a0-al)] ; 

goto  1; 

4  Results 

The  problem  we  solve  with  d/dt  is  the  following:  given  some  initial  low-energy 
orbit  (more  precisely,  a  connected  set  of  orbits)  is  there  a  sequence  of  switchings 
that  brings  the  system  to  its  target,  a  higher-energy  set  of  orbits?  This  problem 
is  essentially  a  controller  synthesis  problem  for  the  eventuality  specification,  un¬ 
like  the  safety  controller  synthesis  that  we  have  treated  in  [ABD+00].  We  are 
interested  in  reaching  the  desired  orbit  with  the  least  number  of  mode  switchings. 

We  illustrate  informally  the  synthesis  procedure  that  we  employ  in  order  to 
derive  the  switching  controller.  Consider  an  initial  set  of  orbits  characterized  by 
the  rectangle  (in  the  92)  space)  P  =  [0.7  x  0.9]  x  [0.01, 0.02]  at  state  S3  and  a 
goal  orbit  characterized  hy  F  ^  [1.05, 1.3]  x  [0.01, 0.02]  at  the  same  state.  Starting 
from  the  inital  set  (s,  P)  we  calculate,  in  a  breadth-first  manner,  all  its  successors, 
i.e.  continuous  successors,  and  then,  via  intersection  with  the  guards,  the  discrete 
successors.  We  continue  until  at  some  level  k  of  the  search  tree,  there  is  one  or 
more  paths  having  a  leaf  (s,  Q)  such  that  Q  intersects  F.  The  search  graph  of  the 
first  iteration  is  shown  in  Figure  3  and  there  are  two  intersections  with  the  goal 
orbit  after  4  transitions,  along  the  paths  S3,  S2,  S3,  S2,  S3  and  S3,  S2,  si,  S2,  S3.  For 
every  such  path  we  do  backward  reachability  analysis  to  find  the  predecessors 
of  the  goal  orbit  at  every  node  and,  in  particular,  the  subset  of  P  from  which 
the  goal  can  be  reached  by  taking  the  k  transitions  that  correspond  to  the  path. 
This  information  is  also  used  to  derive  the  controller  by  restricting  the  guards. 
In  our  example  we  conclude  that  points  satisfying  q2  G  [0.7552,0.9]  can  reach 
the  goal  orbit  by  following  the  sequence  S3,  S2,  S3,  S2i  ^3  and  those  satisfying 
q2  G  [0.7152,0.9]  can  do  it  following  the  sequence  S3,  S2,  si,  S2,  S3.  Note  that 
from  the  interval  [0.7552,  0.9]  both  sequences  can  be  taken. 

If  not  all  points  in  P  are  “covered”  by  the  A:-length  sequences  found  in  the  first 
iteration,  we  restart  the  procedure  from  (s,  P')  where  P'  C  P  is  the  subset  of  P 
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consisting  of  the  points  not  covered  yet.  In  our  example  P'  consists  of  the  points 
satisfying  G  [0.7, 0.7152] .  In  the  second  iteration  we  find  out  that  the  goal  orbit 
can  be  reached  from  any  point  in  P'  by  either  one  of  the  three  6-transition  se¬ 
quences  S3,S2,53,S2,S3,S2,S3,  S3,  S2,  S3,  S2,  Sj,  S2,  S3  and  S3,S2,Si,S2,Si,S2,S3, 

and  this  concludes  the  computation.  The  fact  that  ^2  does  not  matter  here  is 
particular  to  this  example  —  with  other  sets  of  parameters  the  partition  of  the 
initial  set  did  involve  conditions  on  92-  The  reachable  states  which  correspond 
to  the  discovery  of  the  sequence  S3,  S2,  si,  S2,  si,  S2,  S3  in  the  second  iteration  are 
depicted  in  Figure  4  and  5. 


Fig.  3.  The  first  iteration  of  the  search  tree.  The  goal  orbits  were  first  reached  after 
4  transitions  along  two  paths  of  the  tree. 


5  Conclusion 


We  have  investigated  a  new  methodology  for  designing  hybrid  controllers  which 
is  partially-supported  by  our  reachability  analysis  tool  d/dt.  Like  [ABD+00]  and 
[TLSOO]  this  work  explores  the  contribution  of  the  hybrid  automaton  model  to 
the  alternative  formulation  and  solution  of  problems  in  switching-based  control. 
In  this  paper  we  have  treated  an  interesting  and  open  problem  in  robot  control 
and  provided  a  partial  solution.  To  improve  the  performance  of  the  algorithm,  we 
plan  to  investigate  other  search  procedures  (backward  computation  and  heuristic 
search)  and  validate  our  results  via  simulation. 
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S2  S2  Si 

Fig.  4.  Computation  of  reachable  states  for  the  sequence  S3,S2,si,S2.  On  the  left  we 
see  the  reachable  set  at  mode  Si  while  at  the  right  we  show  the  intersecion  with  the 
guard  from  Si  to  Sj. 


On  Hybrid  Control  of  Under- Actuated  Mechanical  Systems 


87 


<72 


Q2 


q2 


Q2 


Si 


Si  ->  S2 


q2 


Q2  , 


S2 


52  S3 


02 


S3 


Fig.  5.  Computation  of  reachable  states  for  the  sequence  S3,  S2,  si,  S2,  si,  S2,  S3  contin¬ 
ued  from  Figure  4. 
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Abstract.  In  this  paper  we  develop  an  algorithm  for  solving  the  reach¬ 
ability  problem  of  two-dimensional  piece- wise  rectangular  dijfferential  in¬ 
clusions.  Our  procedure  is  not  based  on  the  computation  of  the  reach-set 
but  rather  on  the  computation  of  the  limit  of  individual  trajectories.  A 
key  idea  is  the  use  of  one-dimensional  affine  Poincare  maps  for  which  we 
can  easily  compute  the  fixpoints.  As  a  first  step,  we  show  that  between 
any  two  points  linked  by  an  arbitrary  trajectory  there  always  exists  a 
trajectory  without  self-crossings.  Thus,  solving  the  reachability  problem 
requires  considering  only  those.  We  prove  that,  indeed,  there  are  only 
finitely  many  “qualitative  types”  of  those  trajectories.  The  last  step  con¬ 
sists  in  giving  a  decision  procedure  for  each  of  them.  These  procedures 
are  essentially  based  on  the  analysis  of  the  limits  of  extreme  trajectories. 
We  illustrate  our  algorithm  on  a  simple  model  of  a  swimmer  spinning 
around  a  whirlpool. 


1  Introduction 

One  of  the  main  research  areas  in  hybrid  systems  is  reachability  analysis.  It 
comprises  two  (closely  related)  issues,  namely,  the  study  of  decidability  and  the 
development  of  algorithms.  Most  of  the  proved  decidability  results  are  based 
on  the  existence  of  a  finite  and  computable  partition  of  the  state  space  into 
classes  of  states  which  are  equivalent  with  respect  to  reachability.  This  is  the 
case  for  timed  automata  [2],  and  classes  of  rectangular  automata  [12]  and  hybrid 
automata  with  linear  vector  fields  [15].  Except  for  timed  automata,  these  results 
rely  on  stringent  hypothesis  such  as  the  resetting  of  variables  along  transitions. 

Although  analysis  techniques  based  on  the  construction  of  a  finite  partition 
have  been  proposed  [7],  mainly  all  implemented  computational  procedures  resort 
to  (forward  or  backward)  propagation  of  constraints,  typically  (unions  of  convex) 
polyhedra  or  ellipsoids  [1,3,6,9,11,14].  In  general,  these  techniques  provide  semi¬ 
decision  procedures,  that  is,  if  the  given  final  set  of  states  is  reachable,  they  will 
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terminate,  otherwise  they  may  fail  to.  This  is  a  property  of  the  techniques,  not 
of  the  problem,  that  is,  it  does  not  imply  that  the  reachability  problem  itself  is 
undecidable,  but  only  that  they  do  not  implement  a  decision  procedure  for  it.  In 
other  words,  these  algorithms  may  be  unsuccessful  (i.e.,  not  terminate)  for  cer¬ 
tain  classes  of  systems  for  which  the  reachability  problem  is  indeed  decidable  (by 
other  means).  Nevertheless,  they  provide  tools  for  computing  (approximations 
of)  the  reach-set  for  large  classes  of  hybrid  systems  with  linear  and  non-linear 
vector  fields. 

Maybe  the  major  drawback  of  set-propagation,  reach-set  approximation  pro¬ 
cedures  is  that  they  pay  little  attention  to  the  geometric  properties  of  the  specific 
(class  of)  systems  under  analysis.  To  our  knowledge,  in  the  context  of  hybrid  sys¬ 
tems  there  are  two  lines  of  work  in  the  direction  of  developing  more  “geometric” 
approaches.  One  is  based  on  the  existence  of  (enough)  integrals  and  the  ability 
to  compute  them  all  [7,10].  These  methods,  however,  do  not  necessarily  result 
in  decision  procedures  (they  are  actually  not  meant  to).  The  other,  applica¬ 
ble  to  two-dimensional  dynamical  systems,  relies  on  the  topological  properties 
of  the  plane,  and  explicitly  focuses  on  decidability  issues.  This  approach  has 
been  proposed  in  [16],  There,  it  is  shown  that  the  reachability  problem  for  two- 
dimensional  systems  with  piece-wise  constant  derivatives  (PCD)  is  decidable. 
This  result  has  been  extended  in  [8]  for  planar  piece- wise  Hamiltonian  systems. 
In  [4]  it  has  been  shown  that  the  reachability  problem  for  PCD  is  undecidable 
for  dimensions  higher  than  two. 

In  this  paper  we  develop  an  algorithm  for  solving  the  reachability  problem 
of  two-dimensional  piece- wise  rectangular  differential  inclusions.  As  in  [16],  our 
procedure  is  not  based  on  the  computation  of  the  reach-set  but  rather  on  the 
computation  of  the  limit  of  individual  trajectories.  A  key  idea  is  the  use  of  one¬ 
dimensional  affine  Poincare  maps  for  which  we  can  easily  compute  the  fixpoints. 
The  decidability  result  of  [16]  fundamentally  relies  on  the  determinism  of  PCD 
which  implies  that  planar  trajectories  do  not  intersect  themselves.  This  property 
is  no  longer  true  for  differential  inclusions.  As  a  first  step,  we  show  that  between 
any  two  points  linked  by  an  arbitrary  trajectory  there  always  exists  a  trajectory 
without  self-crossings.  Thus,  solving  the  reachability  problem  requires  consider¬ 
ing  only  those.  We  prove  that,  indeed,  there  are  only  finitely  many  “qualitative 
types”  of  those  trajectories.  The  last  step  consists  in  giving  a  decision  procedure 
for  each  of  them.  These  procedures  are  essentially  based  on  the  analysis  of  the 
limits  of  extreme  trajectories  (which  do  not  cut  themselves). 

2  Simple  Planar  Differential  Inclusions 

A  simple  planar  differential  inclusion  system  (SPDI)  consists  of  a  partition  of 
the  plane  into  convex  polygonal  regions,  together  with  a  differential  inclusion 
associated  with  each  region.  As  an  example  consider  the  problem  of  a  swimmer 
trying  to  escape  from  a  whirlpool  in  a  river. 

Example.  The  dynamics  x  of  the  swimmer  around  the  whirlpool  is  approximated 
by  the  piece-wise  differential  inclusion  defined  as  follows.  The  zone  of  the  river 
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nearby  the  whirlpool  is  divided  into  8  regions  Ri,...,Rs.  To  each  region  Ri 
we  associate  a  pair  of  vectors  (ai,b^)  meaning  that  x  belongs  to  their  positive 
hull:  ai  =  bi  =  (1,5),  aj  =  b2  =  (-l,i),  as  =  (-1,  g)  and  bs  = 
a4  =  b4  =  (-1,-1),  as  =  bj  =  (0,-1),  as  =  be  =  (1,-1),  ay  =  by  =  (1,0), 
ag  =  bs  =  (1, 1).  The  corresponding  SPDI  is  illustrated  in  Fig.  1.  □ 


Fig.  1.  The  SPDI  of  the  swimmer. 


More  formally,  a  SPDI  is  a  pair  7i  =  where  P  is  a  finite  partition  of 

the  plane  into  convex  polyhedral  sets,  and  for  each  P  e  P,  (l>{P)j  also  denoted 
by  is  the  set  of  all  linear  combinations  x  =  o:  ap  +  /?  bp,  with  a,/?  >  0, 
and  o;  +  >  0,  of  two  vectors  ap  and  bp,  such  that  ap  •  bp  <0,  where  •  is  the 

scalar  product  and  ap  =  (02,  — ^i)  is  the  clockwise  rotation  of  ap  by  the  angle 
~  (notice  that  ap  •  ap  =0). 

Let  E{P)  be  the  set  of  edges  of  P,  that  is,  the  set  of  open  segments  forming 
the  boundary  of  P,  and  V (P)  be  the  set  of  vertices  in  the  boundary  of  P.  We 
say  that  e  is  an  entry  of  P  if  for  all  x  G  e  and  for  all  c  G  ^(P),  x  +  ce  G  P 
for  some  e  >  0.  We  say  that  e  is  an  exit  of  P  if  the  same  condition  holds  for 
some  €  <  0.  We  denote  by  m(P)  C  E{P)  the  set  of  all  entries  of  P  and  by 
out(P)  C  E{P)  the  set  of  all  exits  of  P.  In  general,  E(P)  ^  i-n{P)  U  out(P). 
We  say  that  P  is  a  good  region  iff  all  the  edges  in  P(P)  are  entries  or  exits, 
that  is,  E{P)  =  in{P)  U  out(P).  Notice  that,  if  P  is  a  good  region,  then  for  all 
e  G  E{P),  the  director  vector  of  e  does  not  belongs  to  (^(P)  (Fig.  2).  Hereinafter, 
we  assume  that  all  regions  are  good  regions.  Let  x  G  F (P)  be  a  common  vertex 
of  two  edges  e  and  e'.  x  is  an  entry  point  to  P  if  both  e  and  e'  are  entry  edges; 
it  is  an  exit  point  if  both  e  and  e'  are  exit  edges.  In  fact,  vertices  can  be  seen  as 
a  particular  kind  of  edges,  with  exactly  one  point.  In  what  follows  the  term  edge 
will  be  understood  as  belonging  to  the  set  EV{P)  =  E{P)  U  V{P).  If  needed, 
the  difference  between  edge  and  vertex  will  be  explicitly  specified. 

A  trajectory  in  some  interval  [0,T]  C  R,  with  initial  condition  x  =  Xq,  is  a 
continuous  and  almost-everywhere  (everywhere  except  on  finitely  many  points) 
derivable  function  ^(•)  such  that  ^(0)  =  Xq  and  for  all  t  G  (0,T),  if  ^(t)  G 
P  \  EV{P),  then  ^{t)  is  defined  and  ^{t)  G  (t>(P)- 

The  point-to-point  reachability  problem  for  R,  is  the  following:  Given  x,  x'  G 
R^,  is  there  a  trajectory  ^  and  t  >  0  such  that  ^(0)  =  x  and  ^(t)  =  x'?.  If  the 
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(a)  (f>) 


Fig.  2.  a)  A  good  region,  b)  A  bad  region  (es  0  m{P)  U  out(P)). 


answer  is  yes,  we  say  that  x'  is  reachable  from  x.  The  edge-to-edge  reachability 
problem  is  the  following:  Given  two  edges  e  and  e'  of  H,  is  there  x  G  e  and  x'  €  e' 
such  that  x'  is  reachable  from  x?  The  region-to-region  reachability  problem  is 
defined  similarly. 

3  Properties  of  Trajectories 

W.l.o.g.  we  will  consider  in  what  follows  that  ^(0)  G  e  for  some  edge  e.  The 
trace  of  a  trajectory  ^  is  the  sequence  r($)  =  xqXi  ...  of  the  intersection  points 
of  ^  with  the  set  of  edges,  that  is,  x*  G  $  O  \JEV{P)  for  all  P  gV.  The  edge 
signature  of  f  is  the  sequence  (t(^)  =  eoCi ...  of  traversed  edges,  that  is,  x^  G  e*. 
The  region  signature  of  ^  is  the  sequence  p(^)  =  PqPi  . . .  of  traversed  regions, 
that  is,  Ei  G  in{Pi). 

Let  ^  be  a  trajectory  whose  trace  is  r($)  =  Xq  . . .  Xa;.  Let  0  =  to  <  h  < 
...<  tk  he  such  that  =  x^.  Since  ^  is  continuous  and  derivable  in  the 
interval  (ti,ti+i),  there  exists  a  unique  trajectory  with  ^{ti)  =  ^(U)  for  all 
i  G  [0,  A:  - 1],  such  that  the  derivative  is  constant  in  the  interval  (tj,  U^i).  That 
is, 

Proposition  1.  For  every  trajectory  ^  there  exists  a  trajectory  with  the  same 
initial  and  final  points,  and  edge  and  region  signatures,  such  that  for  each  Pi 
in  the  region  signature,  there  exists  Ci  G  such  that  i'{t)  =  Ci  for  all 

i  €  (ti,  ti+l)- 

Hence,  in  order  to  solve  the  reachability  problem  it  is  enough  to  consider  trajec¬ 
tories  having  piecewise  constant  slopes.  Notice  that,  however,  such  slopes  need 
not  be  the  same  for  each  occurrence  of  the  same  region  in  the  region  signature. 
Hereinafter,  we  use  the  word  “trajectory”  to  mean  trajectories  whose  derivatives 
are  piecewise  constant. 

Consider  a  region  P  and  let  c  €  0(P).  The  mapping  Q  :  ^  R,  defined 

as  J7(x)  =  X  ■  c,  assigns  to  every  x  e  a  value  proportional  to  the  length  of 
the  projection  of  the  vector  x  on  the  right  rotation  of  c  (see  [4]).  Indeed,  the 
ordering  is  given  by  the  direction  of  c  and  one  can  easily  see  that  the  relation 
defined  as  xi  ::<  X2  if  i?(xi)  <  12 (X2),  is  a  dense  linear  order  on  iii(P)  and 
out(P)  (Fig.  3).  We  use  -<  to  denote  the  strict  variant  of  ^  and  say  that  ei  -<  62 
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iff  ei  ^  62  and  xi  X2  for  every  Xi  G  ei,X2  G  62-  For  example,  in  Fig.  3  we 
have  61  -<  62  ^  63.  Notice  that  the  order  does  not  depend  on  the  choice  of  c. 


out(P) 


Fig.  3.  Ordering:  Xi  :<  X2. 


We  say  that  a  trajectory  ^  crosses  itself  if  there  exist  t  ^  t'  such  that  ^(t)  = 
If  a  trajectory  does  not  cross  itself,  the  sequence  of  consecutive  intersection 
points  with  in{P)  or  out(P)  is  monotone  with  respect  to  :<,  That  is,  for  every 
three  points  xi,  X2  and  X3  (visited  in  this  order),  if  xi  ^  X2  X3  the  trajectory 
is  a  “counterclockwise  expanding  spiral”  (Fig.  4(a))  or  a  “clockwise  contracting 
spiral”  (Fig.  4(b))  and  if  X3  X  X2  Xi,  the  trajectory  is  a  “counterclockwise 
contracting  spiral”  (Fig.  4(c))  or  a  “clockwise  expanding  spiral”  (Fig.  4(d)).  On 
the  other  hand,  if  the  sequence  of  intersections  points  with  in(P)  or  out(P)  is 
monotone  (both  increasing  or  both  decreasing),  the  trajectory  does  not  cross 
itself. 

Lemma  1.  For  every  trajectory  if^  does  not  cross  itself,  then  for  every  edge 
e,  the  sequence  r{^)  He  is  monotone  (with  respect  to  -<). 


Fig.  4.  Spirals. 


Now  suppose  that  the  trajectory  $  with  trace  r($)  =  xq  . . .  X/  crosses  itself  once 
inside  the  region  P.  Let  ei, 62  G  in{P)  be  the  input  edges  and  6^, 62  G  out(P)  be 
the  output  ones.  Let  x  =  x^  G  ei  and  y  ~  Xj  G  62,  with  i  <  j,  he  the  points  in 
t(^)  the  first  and  the  second  times  ^  enters  P,  and  let  x'  =  Xj+i  G  62  and  y'  = 
Xj+i  G  e'l  be  the  corresponding  output  points.  Let  Cx,Cy  G  <p(P)  =  be  the 
derivatives  of  ^  in  the  time  intervals  (ii,ii+i)  and  {tj,tj^i),  respectively.  Indeed, 
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Cx  and  Cy  are  the  director  vectors  of  the  segments  xx'  and  yy',  respectively 
(Fig.  5(a)).  _ 

Consider  now  the  segment  xy'.  Notice  that  the  director  vector  of  this 
segment  can  be  obtained  as  a  positive  combination  of  the  vectors  and  Cy. 
Thus,  e  Hence,  there  exists  a  trajectory  that  does  not  cross  itself  in 

P  having  a  trace  r{^')  =  xq  . . .  xy' . . .  x/  (Fig.  5(b)).  Notice  that  the  result  also 
works  for  the  degenerate  case  when  the  trajectory  crosses  itself  at  an  edge  (or 
vertex). 


Fig.  5.  Obtaining  a  non-crossing  trajectory 


If  the  trajectory  ^  crosses  itself  more  than  once  in  region  P,  then  the  number 
of  times  the  trajectory  obtained  by  cutting  away  the  loop  (Fig.  5(c)),  crosses 
itself  in  P  is  strictly  smaller  than  the  number  of  times  ^  does  it  (see  Fig.  6). 
After  replacing  xx'  and  yy'  by  the  intersection  q  of  ^  and  ^  disappears. 
If  the  new  segment  of  line  xy'  crosses  another  segment  zz'  (say  at  a  point  t), 
then  zz'  necessarily  crosses  either  xx'  (at  r)  or  3^  (at  s)  -or  both-,  before  the 
transformation.  The  above  is  due  to  the  fact  that  if  zz'  crosses  one  side  of  the 
triangle  xy'q  then  it  must  also  cross  one  of  the  other  sides  of  the  triangle,  say 
at  r.  Thus,  no  new  crossing  can  appear  and  the  number  of  crossings  in  the  new 
configuration  is  always  less  than  in  the  old  one. 


Fig.  6.  The  number  of  crossings  decreases:  (a)  Before  (3  crossings);  (b)  After  (1  cross¬ 
ing). 
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Lemma  2.  For  every  trajectory  ^  that  crosses  itself  at  least  once,  there  exists 
a  trajectory  with  the  same  initial  and  final  points  of  ^  having  a  number  of 
self- crossings  strictly  smaller. 

The  above  result  follows  from  a  straightforward  inductive  reasoning,  as  well  as 
the  following  one. 

Proposition  2.  If  there  exists  an  arbitrary  trajectory  from  points  Xq  G  gq  to 
Xf  ^  Cf  then  there  always  exists  a  non-crossing  trajectory  between  them. 

Hence,  in  order  to  solve  the  reachability  problem  we  only  need  to  consider  non¬ 
crossing  trajectories  with  piecewise  constant  derivatives.  In  what  follows,  we  only 
deal  with  trajectories  of  this  kind. 

4  Properties  of  Edge  Signatures 

Let  ^  be  a  trajectory  with  trace  r(^)  —  Xq  . . .  Xp,  edge  signature  cr(^)  =  eo  . . .  Cp, 
and  region  signature  /?($)  =  Po...Pp.  An  edge  e  is  said  to  be  abandoned  by 
$  after  position  z,  if  ==  e  and  for  some  j,  fc,  i  <  j  <  k,  Pj  ..  .Pk  forms  a 
region  cycle  and  e  ^  {ej+i,.. .  ,ek}.  Since  trajectories  are  finite  we  should  add 
the  trivial  case  when  e  ^  ej  for  all  j  >  i. 

Lemma  3  (Claim  2  in  [4]).  For  every  trajectory  f  and  edge  e,  ife  is  abandoned 
by  ^  after  position  i,  e  will  not  appear  in  (j(^)  at  any  position  j  >  i. 

Given  a  sequence  s,  we  use  notations  first{s)  and  last{s)  for  the  first  and  last 
elements  of  the  sequence  respectively.  €  denotes  the  empty  sequence  An  edge 
signature  <j(^)  can  be  canonically  expressed  as  a  sequence  of  edges  and  cycles  of 
the  form  crc(0  =  7-151^252^  . .  .rnS^^rn+i,  where 

1.  For  alH  €  [1,  n  -h  1],  is  a  sequence  of  pairwise  different  edges; 

2.  For  all  i  €  [l,n],  Si  is  a  simple  cycle  (i.e.,  without  repetition  of  edges) 
repeated  ki  times; 

3.  For  all  i  €  [l,n  -  1],  first{ri+i)  /  first{si)  if  ri_^i  e,  otherwise 
first(si+i)  ^  first{si); 

4.  For  all  i  G  ii  ri  ^  e  then  last{ri)  =  last{si)] 

5.  rn+i  ^  Moreover,  =  first{sn)  if  o'(^)  ends  in  a  loop  and 

first{rn+i)  /  first{sn)  otherwise. 

This  canonical  representation  can  be  obtained  as  follows.  Let  =  ei . . .  Cp-iCp 
be  an  edge  signature.  Starting  from  ep_i  and  traversing  backwards,  take  the 
first  edge  that  occurs  the  second  time.  If  there  is  no  such  edge,  then  trivially 
the  signature  can  be  expressed  in  a  canonical  way  and  we  are  done.  Otherwise, 
suppose  that  the  edge  ej  occurs  again  at  position  i  (i.e.  =  Cj  with  i  <  j), 

thus  C7c(0  =  wsr,  where  w,  s  and  r  are  obtained  as  follows,  depending  on  the 
repeated  edge: 

w  =  eQ...ei 
•5  = 

r  =■  ejj^\ . . ,  Cp—i 
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Clearly  r  is  not  a  cycle  and  s  is  a  simple  cycle  with  no  repeated  edges.  We 
continue  the  analysis  with  w.  Let  km  =  max{l  \  sMs  a  suffix  of  w}.  Thus, 
^c{C)  =  w's^r  with  w'  =  eo...eh  (a  prefix  of  w)  and  /c  =  /c^n  +  1*  We  repeat 
recursively  the  procedure  above  with  w'.  Adding  the  edge  Cp  to  the  last  r  (at  the 
end)  we  obtain  crc(^)  =  . .  .rnS^^Vn+i  that  is  a  canonical  representation  of 

signature  a. 

Let  us  define  the  type  of  a  signature  a  as  type{a{^)  =  ri,  si, . . . , 

Notice  that  the  “preprocessing”  (taking  away  the  last  edge  Cp)  is  done  in  order 
to  differentiate  edges  signatures  that  end  with  a  cycle  from  those  that  do  not. 
There  exists  many  other  (maybe  easier)  ways  of  decomposing  a  signature  a  in  a 
canonical  form  (in  particular,  traversing  forward  instead  of  backwards) ,  but  the 
one  chosen  here  permits  a  clearer  and  simpler  presentation  of  the  reachability 
algorithm.  In  fact  in  this  canonical  form,  the  last  visited  edge  in  a  cycle  ei . . . 
is  always  the  last  one  (e^). 

Example.  Let  us  consider  the  following  examples.  Suppose  that  a  =  abcdbcefg 
g^f 9^fhi.  Then,  after  applying  once  the  above  procedure  we  obtain  that  (Jc  = 
'^(52)^^!,  with  w  =  abcdbcef;  S2  =  gef]  rj  =  h.  Applying  the  procedure  once 
more  to  w  we  obtain  w  =  w' {$3)^x2  with  w'  =  —  abc;  S3  =  dbc;  r2  =  ef. 

Putting  all  together  and  adding  the  last  edge  {i)  gives  ac  =  abc{dbcyef(gef)^hi 
with  type  type{a)  =  abc,dbc^ef^gef,hi.  Suppose  now,  that  the  signature  ends 
with  a  cycle:  a  =  abcdbcef  gef  gefgefgef .  In  this  case  we  apply  the  preprocessing 
obtaining  =  w{s2Yri  with  w  =  abcdbce;  S2  =  fge;  ri  =  e.  Applying  the 
procedure  to  w  we  finally  obtain  w  =  w'{s3)^r2  with  w'  —  r3  —  abc;  S3  = 
dbc]  r2  =  e  and  that  gives  cTc  =  abc{dbc)^e{fge)'^f  (adding  /  to  the  end).  □ 

Lemma  4.  The  type  of  a  signature  a,  type{(j),  has  the  following  properties: 

1.  For  every  l<i^j<n-\-l,ri  and  rj  are  disjoint; 

2.  For  every  \  <  i  ^  j  <  n,  Si  and  Sj  are  different; 

3.  If  V  is  a  vertex  appearing  in  type{cr),  then  it  can  only  occur  exactly  once  in 
ri  for  some  1  <  i  <  n  +  I  in  a.  Moreover,  v  ^  last{ri)  unless  i  =  n  +  l. 

Proposition  3.  The  set  of  types  of  edge  signatures  is  finite. 

Thus,  to  solve  the  reachability  problem  we  can  proceed  by  examining  one  by  one 
the  types  of  signatures. 


5  Affine  Operators 

Before  getting  into  the  problem  of  analyzing  types  of  edge  signatures,  we  need 
to  introduce  some  useful  notions. 

An  ajfine  function  /  :  E  M  is  defined  by  a  formula  f{x)  =  ax  +  b  with 
a  >  0.  An  ajfine  multivalued  operator  F  :  R  2^  is  determined  by  two  affine 
functions  fi{x)  and  fu{x)  and  maps  x  to  the  interval  {fi(x),fu{x)),  where  (a,  6) 
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means  (a,  6),  [a,  6],  (a,  6]  or  [a,  6)  :  F{x)  =  fu{x)).  We  use  the  nota¬ 

tion  F  =  (fufu)-  Such  an  operator  can  be  naturally  extended  to  subsets  of 
M:  F{S)  =  UxeS'^(^)-  particular,  if  5  =  {l,u):  F({l,u))  =  {fi{l),fu(u)).  A 
truncated  affine  multi-valued  operator  G  :  M  2®  is  determined  by  an  affine 
multi-valued  operator  F  and  an  interval  (L,  U)  as  follows:  G{x)  =  F{x)n{L,  U). 
Such  operators  can  be  also  extended  to  sets.  We  use  notations  G  =  F  C\  [L^U) 
and  F  —  G. 

Lemma  5  (composition  of  affine  operations).  Affine  functions,  affine 
multi-valued  operators,  and  truncated  affine  multi-valued  operators  are  closed 
under  composition. 

Example.  Let  Gi{x)  =  (22: -f  3,  So: +  5]  and  62(2:)  =  [52:4-2,  72j-|-6]  be  two  (non- 
truncated)  affine  multi-valued  functions,  Gi  =  (5in(l,  6],  and  G2  =  ^2  0  [6,  10) 
their  truncated  versions.  The  truncated  affine  multi-valued  operator  G2  oGi{x) 
is  obtained  as  follows: 

G2  o  Gi{x)  =620  Gi{x)  n  G2{{h  6])  n  [6,  10) 

=  (5(22:  +  3)  -f  2,  7(32:  -h  5)  -b  6]  n  (5  •  1  -h  2,  7  ■  6  -f  6]  n  [6,  10) 

=  (IO2:  -h  17,  2I2:  -b  41]  n  (7,  48]  n  [6,  10) 

=  (lOx  -b  17,  2I2:  -b  41]  n  (7,  10). 

Notice  also  that  for  any  interval  {l,u)  its  image  is  G2  o  Gi{{l,u))  =  {101  -b 
17,  21u-b41)n(7,  10).  □ 

Let  /  be  an  affine  function,  2:0  be  any  initial  point  and  Xn  =  Clearly, 

the  sequence  Xn  is  monotonous,  and  it  converges  to  a  limit  x*  (finite  or  infinite). 
Indeed,  x*  can  be  effectively  computed  knowing  a,  b  and  xq,  as  follows.  If  a  <  1, 
X*  is  the  unique  fixpoint  of  /,  that  is,  ax*  4-6  =  2:*,  which  yields  x*  =  6/(1  —  a). 

If  a  =  1,  2:*  =  —00  if  6  <  0,  2:*  =  00  if  6  >  0,  and  x*  =  2:0,  if  6  =  0.  If  a  >  1, 

let  2:*  =  6/(1  —  a),  then  2:*  =  —00  if  2:0  <  x*  =  00  if  xq  >  2:*,  x*  =  xq  —  x*, 
otherwise.  This  result  can  be  extended  to  multi-valued  affine  functions. 

Lemma  6.  Let  {Io,uq)  be  any  initial  interval  and  {In^Un)  =  -^”((^0?  ^o))-  Then 

1.  The  sequences  In  and  Un  are  monotonous; 

2.  They  converge  to  limits  I*  andu*  (finite  or  infinite),  which  can  be  effectively 
computed. 

Proposition  4.  Let  F  be  truncated  affine  and  I  C  {L,U).  Then  F^{I)  = 
F-{I)f]{L.U). 

6  Computing  the  Successor  Function 

To  solve  the  reachability  problem  for  SPDI,  the  next  step  is  to  provide  a  pro¬ 
cedure  for  computing  the  successors  of  a  point  (and  an  interval),  which  requires 
having  an  effective  representation  of  (rational)  points  and  intervals  on  edges. 
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Let  us  first  introduce  a  one-dimensional  coordinate  system  on  each  edge.  For 
each  edge  e  we  chose  (1)  a  point  on  it  (the  origin)  with  radius- vector  v,  and 
(2)  a  director  vector  e  going  in  the  positive  direction  in  the  sense  of  the  order 
Now  to  characterize  e  we  need  the  coordinates  of  its  extreme  points,  namely, 
€  QU{-oo,  oo}  such  that  e  =  {x  =  v  +  a:e  |  <  x  <  e^}.  That  is,  an  edge 

E  can  be  represented  by  a  triplet  (v,e,  (e^e^)).  If  the  edge  is  a  vertex,  the 
representation  is  simply  (v,  [0, 0]).  Now,  every  point  x  =  v+xe  G  e  is  represented 
by  the  pair  {e,x)  (Fig.7(a)),  and  every  interval  (xi,X2)  C  e  is  represented  as 
(e,  {a:i,a:2)),  where  Xi  =  {e,xi)  and  X2  =  (6,0:2)  (Fig. 7(b)).  Now,  having  fixed 


Fig.  7.  (a)  Representation  of  edges;  (b)  Representation  of  an  interval;  (c)  One-step 
successor. 


a  one-dimensional  coordinate  system  to  represent  points,  the  question  now  is  to 
take  advantage  of  it  to  compute  the  successor  of  a  point  or  an  interval. 

Let  e  =  {e\e^)  G  in{P)  and  e'  =  (e'\e'^)  G  out{P).  For  x  =  (e^x)  and 
c  G  <?!>(P),  we  denote  by  Succ^  ^/(o:)  the  unique  x'  =  {e',x')  such  that  x'  —  x  +  ct 
for  some  t  >  0.  The  point  {e',  x')  is  the  successor  of  (e,  x)  in  the  direction  c  (see 
Fig.7(c)).  Expanding,  v'  +  xV  =  v  +  xe  +  tc.  Multiplying  both  expressions  by  c 
we  obtain  that  (v'+x'e')c  =  v-c  +  xec,  i.e.  x'(e'-c)  =  x(e-c)-f(v  — v')  -c.  Thus, 
x'  =  SucCg  e,(x)  =  +  -c  and  x'  G  (e'^e'“).  Indeed,  putting  a(c)  = 

and  (3{c)  —  •  c  we  have  the  following  result. 

Lemma  7.  The  function  SucCg  g/(x)  =  q;(c)x  +  /?(c)  n  is  truncated 

affine. 

SucCe,e'(2;)  will  denote  the  non~truncated  function  q:(c)x  +  /3(c).  The  notion 
of  successor  can  be  extended  on  all  possible  directions  c  G  </>(F)  and  it  can  be 
applied  to  any  subset  S  C  (e^,  e^)  and  in  particular  to  intervals  {l,u)\ 

Lemma  8.  Let  <t){P)  =  ^  =  (^5^)  ^  Then: 

1.  Succe,e'(a:)  =  Uce^(p)Succ^_e'W  =  (Succ^,e'(®).Succ“e'(a;))  n(e'',e'“); 

2.  Succ,,,,{{l,u))  =  (S^^L'(0,si;^ZH>  n  (e'^e'"). 

The  successor  operator  will  be  used  as  a  building  block  in  the  reachability  al¬ 
gorithm.  It  can  be  naturally  extended  on  edge  signatures:  for  ty  =  ei,  62, . . . ,  e„ 
let 

Succ^(/)  =  SuCCe„_,,e„  o  .  .  .  o  SuCC^.^ea  SuCCe,,ej(/) 
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that  by  Lemma  5  is  truncated  affine.  Notice  that  since  we  use  edge  signatures 
the  semi-group  property  takes  the  following  form. 

Lemma  9,  For  any  edge  signatures  w  and  v  and  an  edge  e,  SucCg^y  o  Succ^e  = 

SUCC^;gn;  . 


Example.  Let  us  come  back  to  the  example  of  the  swimmer  trying  to  escape 
from  a  whirlpool  in  a  river  (see  Fig.  1).  Suppose  that  the  swimmer  is  following 
a  trajectory  with  edge  signature  (ci . . .  eg)*.  It  is  not  difficult  to  find  a  repre¬ 
sentation  of  the  edges  such  that  for  each  edge  =  (0,1).  Besides,  the 

truncated  affine  successor  functions  are: 


SuCCe.e.(®)  =  [|,  n  (0,  1)  SuCQ.esW  = 

Succe,ei+,(x)  =  [x,x]  fl  {0, 1),  for  all  i  €  [3,7]  Succ^jei  W  = 


3  2 

— ,  a:  H - 

10’  ^15 


1 


n(o,i) 

n(o,l) 


The  successor  function  for  the  loop  s  =  ei . . .  eg  is  obtained  by  composition  of 
the  above  functions  as  follows.  Let  us  first  compute 


SuCCejeaesC^^'^)  =  SuCCg^gg  O  SuCCeiez 


A 

2  10’  2  '  15J  '  ■  V''  10 


-ri_  A,|  +  ^ln(o~^,i  +  ^)n(o,i) 


Since  SucCg-e  - ,  ^  for  i  G  [3,  7]  are  the  identity  functions,  we  have  that 


Succ. 


SuCCegei  oSuCCeie2e3(^'^; 


=  [|-^  +  if  +  A  +  5ln(i  +  i5  +  i)n(0,l) 
=  [|-TO.!  +  5ln{il) 


By  Lemma  6  we  have  that  I*  =  and  u*  = 


□ 


7  Reachability  Analysis 

The  algorithm  for  solving  the  reachability  problem  between  two  points  xq  = 
(eo,a:o)  and  X/  =  (e/,^:/)  is  depicted  in  Fig.  8.  The  proofs  of  soundness  and 
termination  are  given  in  the  extended  version  ([5]).  It  works  as  follows. 


Reach.  From  the  section  above  we  know  that  there  exists  a  finite  number  of  type 
signatures  of  the  form  ri,  Si, . . . ,  r„,  Moreover,  the  type  signatures  are 

restricted  to  those  with  cq  =  first{ri)  and  ey  =  last{rn+i).  Given  such  a  set  of 
type  signatures  type(eo,  e/),  the  algorithm  Reach{')  is  guaranteed  to  terminate, 
answering  YES  if  xy  is  reachable  from  Xq  or  NO  otherwise.  Reachability  from 
Xo  to  Xf  with  fixed  signature  w  is  tested  by  the  function  Reach ty pe ^ f  ^  ■ 
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Reachtype.  Let  the  type  w  have  the  form  w  =  n,  5i, . . .  ,r^, 5n,r„+i.  Put 
fi  ~  and  exi  ~  first(ri^i)  if  is  non-empty  and  fi-\-i  otherwise 

(i.e.  eXi  is  the  edge  to  which  the  trajectory  exits  from  the  loop  Si).  Let  us  say 
that  a  type  signature  has  a  loopend  property  if  =  first(sn),  i.e.  signa¬ 
tures  of  type  w  terminate  by  several  repetitions  of  the  last  loop.  This  algorithm 
uses  two  functions:  Test{S,  s,x)  that  answers  whether  x  is  reachable  from  a  set 
5  (represented  as  a  finite  union  of  intervals)  in  the  loop  s  (formally,  whether 
X  G  and  the  function  Exit{S,s^e)  that  for  an  initial  set  5,  a 

loop  s,  and  an  edge  e  (not  in  this  loop)  finds  all  the  points  on  e  reachable  by 
making  s  several  times  and  then  exiting  to  e  (formally,  it  computes  SucCs+eW, 
which  is  always  a  finite  union  of  intervals  ).  Since  we  know  how  to  calculate  the 
successor  of  a  given  interval  in  one  and  in  several  steps  (SucCee/(-)  and  Succ^(-)), 
in  order  to  implement  Test{-)  and  Exit{-)  it  remains  to  show  how  to  analyze 
the  (simple)  cycles  Si  and  eventually  their  continuation.  Both  algorithms  Test{-) 
and  Exit{-)  start  by  qualitative  analysis  of  the  cycle  implemented  in  the  function 
Analyze{I,  s).  This  analysis  proceeds  as  follows. 

Analyze.  The  function  Analyze{I,s)  returns  the  kind  of  qualitative  behavior  of 
the  interval  I  ~  (l^u)  under  the  loop  s.  Let  s  be  a  simple  cycle,  /  =  first{s)  its 

first  edge,  and  /=(/,«)  c  /  an  initial  interval  and  5uccsj{x)  =  SucCsj{x)  n 
{L,U).  The  first  thing  to  do  is  to  determine  the  qualitative  behavior  of  the 
leftmost  and  rightmost  trajectories  of  the  interval  endpoints  in  the  cycle.  This 
can  be  done  without  itera^g  Succ^/.  Indeed,  by  Lemma  6,  we  can  compute  the 

limits  (^1,  Wi)  =  limyi-^oo  Succ^  j((/,  u})  (notice  that  those  are  limits  only  for  the 
non-truncated  operator  Succ),  not  taking  into  account  that  the  edges  are  possible 
bounded  (we  use  Lemma  4)  and  compare  these  limit  points  corresponding  to 
unrestricted  dynamics  with  L  and  U.  There  are  five  possibilities: 

1.  STAY  The  cycle  is  not  abandoned  by  any  of  the  two  trajectories:  L  <  I*  < 
u*  <U. 


function  Reach{xo,Xf) 

R  =  false 

for  each  w  G  type{eo,ef) 

R  =  RV  Reachtype{xQ,Xf,w) 

^  R 

function  R.ecLchf,yp^(^XQ ^  Xf^  uj)  i 

S  =  (^o) 

for  i  =  1  ton  —  1 

S  =  {Exit{S,  Si,exi)) 

if  loopend  (w) 

then  < —  Test{S,  s^^Xf) 

else  < —  xf  G  (£’xit(5,  s„,  ea:„))? 

function  Exit{S,  s,  ex) 

E  =  $ 

for  each  I  G  S  such  that  Succsj{I)  ^  0 
E  =  EU  ExitAnalyze{SuCCsj{I),  S,  Cx) 
<r-E 

function  Test{S,  s,  x) 

R  =  false 

for  each  I  G  S  such  that  SucCg, f{I)  7^  0 
R~  R\/  TestAnalyze(SuCCsj{I),  S,x) 

R 

Fig.  8.  Main  algorithm. 
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2.  DIE  The  right  trajectory  exits  the  cycle  through  the  left  (consequently  the 

left  one  also  exits)  or  the  left  trajectory  exits  the  cycle  through  the  right 
(consequently  the  right  one  also  exits).  In  symbols,  u*  <  L\/  I*  >U. 

3.  EXIT-BOTH  Both  trajectories  exit  the  cycle  (the  left  one  through  the  left 

and  the  right  one  through  the  right):  I*  <  L  Au*  >  U. 

4.  EXIT-LEFT  The  leftmost  trajectory  exits  the  cycle  but  not  the  other:  I*  < 

L<u*  <U, 

5.  EXIT-RIGHT  The  rightmost  trajectory  exits  the  cycle  but  not  the  other: 

L<1*  <U  <u*. 

Exit.  The  function  Exit{S,s,ex)  should  return  SucCs+e2:(‘^)-  Both  the  argu¬ 
ment  S  and  the  result  are  finite  collections  of  intervals.  The  exploration  is  made 
for  each  initial  interval  separately.  Notice  that  the  call  Succ3j{I)  ensures  that 
I  C  {L,U).  All  the  work  for  each  initial  interval  I  is  done  by  the  function 
ExitAnaiyze(I,s,ex)  which  launches  the  Analyze{’)  procedure  described  above 
and  last,  according  to  the  result  of  this  analysis  launches  one  of  five  special¬ 
ized  procedures  ExUstay-,  Exit  left  ^  Exit  rig  ht->  Exit  rot  Hi  Exit  die  which 
calculates  the  exit  set  (Fig.  10). 


function  Search{I,  x) 
while  Found{I,x)  —  NOTYET 
/  =  Succ3,/(/) 

< —  Found{I,  x) 


function  Found{I,  x) 
cases 

a;  e  /  .  ^  yes 

/  =  0  :  ^  NO 

x<lAl^:  ^NO 
X  >  I  Au  I  :  < —  NO 
else  :  ^  NOTYET 


Fig.  9.  Search  and  Found. 


Test.  The  upper-level  structure  is  the  same  as  for  EXIT:  each  initial  interval 
is  treated  separately  by  Test  Analyze,  which  makes  one  turn  of  the  loop,  calls 
Analyze  and  delegates  all  the  remaining  to  one  of  the  five  specialized  functions 
TestsTAY,  Test^EFT,  TestRjcHT,  TestsoTH,  TestoiE  (Fig.  10).  The  five  spe¬ 
cialized  Test  functions  use  the  following  two  procedures  (Fig.  9).  The  function 
Found(I,x)  determines  if  the  current  interval  I  contains  x  (YES),  does  not  con¬ 
tain  X  and  moves  in  the  opposite  direction  (NO),  or  none  of  both  these  cases 
(NOTYET).  Found{I,  x)  uses  the  fact  that  the  sequences  In  and  Un  are  increas¬ 
ing  or  decreasing  (which  can  be  easily  determined  at  the  stage  of  the  preliminary 
analysis  of  the  loop):  I  t  means  that  the  sequence  ■  of  successive  suc¬ 

cessors  of  I  is  increasing  whereas  1 1  means  that  the  sequence  is  decreasing,  and 
similarly  for  u  t  and  u  4-  The  function  Search{lyx)  iterates  the  loop  s  until  the 
previous  function  Found  gives  a  definite  answer  YES  or  NO  (Fig.  9).  It  is  used 
only  when  its  convergence  is  guaranteed. 
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Exit 

Test 

STAY 

function  Exit  stay  {I  ex) 

function  Test  stay  {I,  s,x) 
cases 

T  <  X  <  u*  :  i —  YES 

a;  <  r  A  U  :  —  NO 

rr  >  u*  A  u  t  ’■< —  NO 

else  :  i —  Search{I ,x) 

DIE 

function  ExitDiE{I,s,ex) 
f  =  first{s) 

S'  =  0 

repeat 

I  -  SuCCsf{I) 
S'=5USuCC.,ex(/) 
until  /  =  0 
^  S 

function  TestDiE{I,s,x) 

< —  Search{I  ,x) 

BOTH 

function  Exit  both  {I,  s,  ex) 
^Succ,,ex((T,C/» 

function  TestsoTHa ,  s,x) 
<^xe{L,u)i 

LEFT 

function  Exit  left  {1, 3,  ex) 

< -  SuCCs,exi(L,u)) 

function  TestLEFT{t ,  s,x) 
cases 

X  e  {L,U*)  :  < —  YES 

x<\l,u*)-.  ^NO 

{L,u*)  <x  /\u‘\  :  i —  NO 
else  ;  i —  Search{I,x) 

RIGHT 

Similar  to  the  previous  case. 

Similar  to  the  previous  case. 

Fig.  10.  Exit  and  Test. 


Fig.  11.  Example:  x/  =  (ei,  |)  is  not  reachable  from  xo  =  (ei,  i)  {u*  <  |). 


Example.  Consider  again  the  swimmer.  Let  xq  =  (ei,  be  her  initial  position. 
We  want  to  decide  whether  she  is  able  to  escape  from  the  whirlpool  and  reach 
the  final  position  x/  =  (ei,  |).  Recall  that  T  =  =  -i  and  u*  =  -ip  =  I. 

^  1—  2  ^  1—  ^  3 

Thus,  by  the  Analyze  function  we  know  that  the  cycle  behaves  as  an  Exit-LEFT 
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and  applying  the  function  Test  left  we  obtain  that  x/  =  (^i,  |)  is  not  reachable 
from  Xo  =  (ei,  because  we  have  that  u  f  and  u*  <  Xf  {-  <  j)  (Fig.  11).  □ 

Prom  all  the  results  above  we  have  the  following  theorem. 

Theorem  1  (Point-to-Point  Reachability).  The  point-to-point,  edge-to- 
edge  and  region-to-region  reachability  problems  for  SPDI  systems  are  decidable. 

□ 


8  Concluding  Remarks 

We  have  presented  an  algorithm  for  solving  the  reachability  problem  for  simple 
planar  differential  inclusions.  The  novelty  of  the  approach  for  the  domain  of  Hy¬ 
brid  System  is  the  combination  of  two  techniques,  namely,  the  representation  of 
the  two-dimensional  continuous  dynamics  as  a  one-dimensional  discrete  system 
(due  to  Poincare),  and  the  characterization  of  the  set  of  qualitative  behaviors  of 
the  latter  as  a  finite  set  of  types  of  signatures. 

One  possible  direction  of  future  work  is  to  try  to  apply  the  same  method  for 
solving  the  parameter  synthesis  problem  for  SPDFs,  that  is,  for  any  two  points, 
xo  and  X/,  assign  a  constant  slope  cp  G  (t){P)  to  every  region  P  such  that  x/  is 
reachable  from  xq,  or  conclude  that  such  an  assignment  does  not  exist.  Clearly, 
the  decidability  of  the  reachability  problem  does  not  imply  the  decidability  of 
the  parameter  synthesis  one. 

Another  question  that  naturally  arises  is  decidability  of  the  reachability  prob¬ 
lem  for  hybrid  automata  whose  locations  are  equipped  with  SPDFs.  We  can  cer¬ 
tainly  find  (stringent)  conditions,  such  as  planarity  of  the  automaton,  “memory¬ 
less”  resets,  etc.,  under  which  decidability  follows  almost  straightforwardly  from 
the  decidability  of  SPDFs.  On  the  other  hand,  it  is  not  difficult  to  see  that 
this  problem,  without  such  conditions,  is  equivalent  to  deciding  whether  given 
a  piece- wise  linear  map  /  on  the  unit  interval  and  a  point  x  in  this  interval, 
the  sequence  of  iterates  x,  f(x),  /(/(x)),  and  so  on,  reaches  some  point  y.  This 
last  question  is  still  open  [13].  And  last  but  not  the  least,  another  interesting 
issue  is  the  complexity  analysis  of  the  algorithm.  It  should  be  based  on  counting 
all  “feasible”  types  of  signatures.  Our  finiteness  argument  of  lemma  4  gives  a 
doubly  exponential  estimation,  which  can  certainly  be  improved. 
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Abstract.  The  behavior  of  the  run  of  an  impulse  differential  inclusion, 
and,  in  particular,  of  a  hybrid  control  system,  is  “summarized”  by  the 
“  initialization  map”  associating  with  each  initial  condition  the  set  of 
new  initialized  conditions  and  more  generally,  by  its  “substratum” ,  that 
is  a  set-valued  map  associating  with  a  cadence  and  a  state  the  next 
reinitialized  state.  These  maps  are  characterized  in  several  ways,  and 
in  particular,  as  “set- valued”  solutions  of  a  system  of  Hamilton- Jacobi 
partial  differential  inclusions,  that  play  the  same  role  than  usual 
Hamilton-Jacobi-Bellman  equations  in  optimal  control. 

Keywords:  hybrid  control,  impulse  control,  differential  inclusion,  vi¬ 
ability,  run,  execution,  periodic,  cadenced  run,  equilibrium,  Kakutani 
Theorem,  contingent  cone,  Marchaud  map. 


1  Introduction 

Impulse  differential  inclusions,  and  in  particular,  hybrid  control  systems,  are  de¬ 
fined  by  a  differential  inclusion  (or  a  control  system)  and  a  reset  map.  A  run  of  an 
impulse  differential  inclusion  is  defined  by  a  sequence  of  cadences,  of  reinitialized 
states  and  of  motives  describing  the  evolution  along  a  given  cadence  between  two 
distinct  consecutive  impulse  times,  the  value  of  a  motive  at  the  end  of  a  cadence 
being  reset  as  the  next  reinitialized  state  of  the  next  cadence. 

A  first  advantage  of  introducing  impulse  differential  inclusions  is  to  sum¬ 
marize  the  usually  protracted  description  of  an  hybrid  system^  by  only  two 
set-valued  maps  F  —  the  right-hand  side  of  the  differential  inclusion  governing 
the  continuous  evolution  of  a  hybrid  system  —  and  describing  the  reset  map 
reinitializing  the  system  when  required  and  a  constrained  set  K  inside  which 
the  evolution  of  the  “run”  or  “execution”  must  remain.  Hence,  for  instance,  the 
existence  of  a  run  of  an  hybrid  system  for  every  initial  set  becomes  a  viability 
problem  of  an  adequate  auxiliary  subset  under  an  impulse  differential  inclusion, 
that  can  be  characterized  elegantly  end  efficaciously. 

^  See  for  instance  among  many  papers  and  books  [13,  Branicky,  Borkar  &  Mitter],  [12, 
Bensoussan  h  Menaldi],  [17,18,  Matveev  &  Savkin]  and  [20,  Shaft  &  Schumacher]. 

M.D.  Di  Benedetto,  A.  Sangiovanni-rVincentelli  (Eds.):  HSCC  2001,  LNCS  2034,  pp.  105-118,  2001. 
(c)  Springer- Verlag  Berlin  Heidelberg  2001 
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The  behavior  of  the  run  is  “summarized”  by  the  “  initialization  map”  U  := 
^{F,R)  associating  with  each  initial  condition  xq  ^  K  the  set  of  new  initialized 
conditions  xi  e  R{x{~ti))  when  x{-)  ranges  over  the  set  of  solutions  to  the 
differential  inclusion  x'  €  F{x)  viable  in  K  until  they  reach  R~^{K)  at  time 
ti  >0atx{-ti)  gR-\K). 

Indeed,  the  sequence  of  successive  initial  conditions  x^  of  a  viable  run  a:(') 
of  the  impulse  differential  inclusion  (F,  R)  —  constituting  the  “discrete  compo¬ 
nent  of  the  run”  —  is  governed  by  the  discrete  system  Xn  G  H  K 

starting  at  rro.  The  knowledge  of  the  sequence  of  initialized  states  Xn  allows  us 
to  reconstitute  the  “continuous  component”  of  the  run  by  solving  the  differen¬ 
tial  inclusion  x'  G  F(x)  starting  at  each  reinitialized  state  x^  and  satisfying  the 
end-point  condition  x^+i  G  R{x{~tn+i)),  which  exists  thanks  to  the  definition 
of  the  map  . 

Assume  for  a  while  that  the  impulse  differential  inclusion  is  actually  an  im¬ 
pulse  differential  equation  (/,  r)  where  the  maps  /  and  r  are  single- valued  and 
that  the  initialization  map  is  single- valued  and  differentiable.  Then  we  shall 
prove  that  the  initialization  map  is  a  solution  to  the  system  of  first-order  partial 
differential  inclusions 


Vi  =  ^ 


3=1 


dui{x) 

dxi 


fj{x)  =  0 


or,  in  a  more  compact  form,  0  =  satisfying  the  “condition” 

VxGKnr~^{K)^  r{x)  =  u(x) 

Actually,  we  shall  extend  this  result  to  general  impulse  differential  inclusions  by 
characterizing  the  initialization  map  as  a  generalized  (set- valued)  solu¬ 

tion  a  Frankowska  solution  —  to  the  system  of  first-order  partial  differential 
inclusions 

0  ^ 

satisfying  the  “condition” 


VxeKnR-\K),  R{x)  C  U{x) 

These  are  indeed  really  Dirichlet  boundary  condition  whenever  the  reset  map 
R  is  defined  only  on  the  boundary  dK  of  a  closed  subset  K  and  maps  OK  into 
the  interior  oi  K.  In  this  case,  resetting  initial  conditions  happens  only  when 
the  continuous  evolution  of  the  state  governed  by  the  differential  inclusion  or 
the  control  system  is  about  to  leave  the  domain  K.  Hence  the  reset  map  assigns 
new  initialized  states  in  the  interior  of  K. 

We  shall  introduce  more  generally  another  set- valued  map  summarizing  the 
behavior  of  an  impulse  differential  inclusion,  called  the  substratum,  that  is  the 
topic  of  this  paper. 
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Outline:  We  begin  by  giving  our  definition  of  impulse  differential  inclusions. 
We  then  recall  the  characterization  of  viable  subsets  under  an  impulse  differential 
inclusion  an  derive  from  it  a  necessary  and  sufficient  condition  for  the  existence 
of  solutions  to  hybrid  differential  inclusions.  Then,  we  devote  the  next  section  to 
the  graphical  properties  of  the  initialization  map  U  and  we  derive  its  properties 
from  the  general  properties  of  viable-capture  basins  of  a  target  by  a  differential 
inclusion. 

In  the  last  section,  we  translate  the  Prankowska  characterization  of  viable- 
capture  basins  in  terms  of  kinds  of  systems  of  first-order  Hamilton- Jacobi  partial 
differential  equations  characterizing  the  substratum  the  solutions  of  which  are 
the  initialization  maps  and  the  substratum. 


2  Impulse  Differential  Inclusions 

“Hybrid  control  systems”,  as  they  are  called  by  engineers,  or  “multiple-phase 
dynamical  economies”,  as  they  are  called  by  economists  (see  for  instance  [16, 
Day]),  or  “Integrate  and  Fire”  models  in  neurobiology  (see  for  instance  [14, 
Brettej)  —  may  be  regarded  as  impulse  differential  inclusions. 

Here,  X  :=  and  Y  :=  denote  finite  dimensional  vector  spaces.  Let 
f  :  X  xY  (->Xbea  single- valued  map  describing  the  dynamics  of  a  control 
system  and  P  :  X  ^  Y  the  set- valued  map  describing  the  state-dependent 
constraints  on  the  controls. 

First,  any  solution  to  a  control  system  with  state- dependent  constraints  on 
the  controls 

(i)  x'{t)  =  f(x{t),u{t)) 

\  ii)  u{t)  G  P{x{t)) 

can  be  regarded  as  a  solution  to  the  differential  inclusion  x'{t)  G  F{x{t))  where 
the  right  hand  side  is  defined  by  F(x)  :=  f{x,P{x))  :=  {f{x,u)}uep{x)- 

Therefore,  from  now  on,  as  long  as  we  do  not  need  to  implicate  explicitly  the 
controls  in  our  study,  we  shall  replace  control  problems  by  differential  inclusions. 

We  shall  say  that  K  is  locally  viable  under  F  if  from  every  x  G  K  starts  a 
solution  a;(-)  to  the  differential  inclusion  x^  G  F{x)  viable  in  K  on  the  nonempty 
interval  [0,ra;[  in  the  sense 


vtG  [o,r,[,  x{t)  gk 

and  that  K  is  viable  if  we  can  take  =  +oo.  It  is  locally  backward  invariant 
under  F  if  for  every  to  +oo[,  x  G  K,  for  all  solutions  a:(-)  to  the  differential 
inclusion  x'  G  F{x)  arriving  at  x  at  time  to,  there  exists  s  G  [0,to[  such  that  a:(') 
is  viable  in  K  on  the  interval  [s,to],  and  backward  invariant  if  we  can  take  s  =  0. 
We  denote  by 


Graph(F)  :=  {{x,y)  eXGY\yG  F{x)} 


the  graph  of  a  set- valued  map  F  :  X  Y  and  Dom(F)  :=  {x  G  X\F(x)  ^  0} 
its  domain. 
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Let  us  set  x{~ t)  x{t)  when  a:(-)  is  defined  on  some  interval  [t—rf,  t[ 

where  ??  >  0,  and,  for  consistency  purposes,  x{s)  =  x{~t)  ii  s  =  t.  An  impulse 
differential  inclusion  (and  in  particular,  an  impulse  control  system)  is  described  by 
a  pair  (F^R),  where  the  set- valued  map  F  :  X  ^  X  mapping  the  state  space 
X  to  itself  governs  the  continuous  evolution  of  the  system  in  K  and  where 

jR,  the  reset  map,  governs  the  discrete  switches  to  new  “initial  conditions”  when 
the  continuous  evolution  is  doomed  to  leave  K. 

Such  a  hybrid  evolution,  mixing  continuous  evolution  “punctuated”  by  dis¬ 
continuous  impulses  at  impulse  times  is  called  in  the  “hybrid  system”  literature 
a  “run"  or  an  "execution". 

Definition  21  Let  us  consider  a  finite  dimensional  vector  space  X,  a  closed 
subset  K  C  X ,  a  set-valued  map  F  :  X  X  and  a  set-valued  map  R  :  X  X , 
regarded  as  a  reset  map.  We  regard  the  pair  {F,  R)  as  the  dynamics  of  an  impulse 
differential  Inclusion. 

A  run  of  the  impulse  differential  inclusion  is  a  map  x(-)  from  [0,  T]  to  X 
if  T  <  -boo  or  from  [0, +oo[  to  X  if  T  =  -hoc  which  is  associated  with  a  non 
decreasing  sequence  T{x{-))  :=  {tn}n>o  o/ impulse  or  switching  times  to  :=  0  < 
1^1  ^  in  ^  ^  T  (depending  on  the  run  a:(-) )  such  that 

1.  ^  Rixifjx)^  if  ^n+l  —  tji, 

2.  or  else,  on  the  interval  \tnffn-\-i\,  x(-)  is  a  solution  to  the  differential  inclusion 
x'  E  F{x)  starting  at  x[tn)  at  time  tn  until  time  tn+i  at  which  we  take 
xitnj^l)  E  R{x(^  ^n+l))* 

We  denote  byrn  :=  tn—tn-i  the  nth  cadence  of  the  run  and  by  Xn{‘)  :=  x{’-\-tn) 
the  nth  motive  of  the  run,  a  solution  to  the  differential  inclusion  x'  E  F{x) 
starting  at  x(t„)  on  the  interval  [0,t„]  and  satisfying  the  end-point  condition 
^n(^n)  ^  R~^ixn+i)^  The  Sequence  of  states  x{tn)  is  called  the  sequence  o/ ini¬ 
tialized  states. 

We  say  that  a  run  3:(')  is  viable  in  K  if  for  any  t>0,  x{t)  E  K. 

At  this  stage,  a  run  a;(-)  can  just  be  a  (discrete)  sequence  of  states  Xn^i  E 
R{xn)  at  a  fixed  time,  or  just  a  (continuous)  solution  a:(-)  to  the  differential 
inclusion  x'  E  F{x),  or  an  hybrid  of  these  two  modes,  the  discrete  and  the 
continuous. 

Hybrid  systems  can  be  regarded  as  instances  of  viable  impulse  differential 
inclusions:  we  refer  to  [2,  Aubin]  or  [11,  Aubin,  Lygeros,  Quincampoix,  Sastry 
&  Seube]  for  more  details  on  that  topic. 

3  The  Substratum  and  the  Initialization  and  Impulse 
Maps 

We  denote  by  <^(2:)  C  C(0, 00;  X)  the  set  of  absolutely  continuous  functions  t  ^ 
x{t)  E  X  satisfying 


for  almost  all  t  >  0,  x'{t)  E  F{x{t)) 
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starting  at  time  0  at  x:  x(0)  =  x  and  by  :  K  C{0,  oo,fC)  the  viable 
solution  map  mapping  an  initial  state  x  £  K  to  the  set  S^[x)  of  solutions  to  the 
differential  inclusion  x'  £  F{x)  starting  aX.  x  £  K  and  viable  in  K. 

The  set- valued  map  5  :  X  C(0,  oo;  X)  is  called  the  solution  map  associated 
with  F. 

We  next  denote  by 

:=  U  {x{t)}  &  := 

x{-)GS^{x)  xec 

the  -viable  reachable  maps  (or  set-valued  flow)  oix  £  K  and  C  c  K  respectively. 
We  set  1?  :=  when  viability  constraints  are  absent. 


Definition  31  We  associate  with  the  dynamics  {F^R)  of  the  impulse  differential 
inclusion  its  substratum  F^  :  R+  xK  ^  K,  that  is  the  set-valued  map 

associating  with  any  (t,  x)  £  R+  x  K  the  subset 

(t^x))  n  K 

of  the  elements  y  £  R{c)  where  c£  C  :=  Kr\R~^{K)  through  which  the  solutions 
to  the  differential  inclusion  x'  £  F{x)  starting  at  x  and  viable  in  K  until  they 
reach  R~'^{K)  at  time  t. 

Knowing  the  substratum  F^j^y  we  introduce 

1.  the  impulse  map 

such  that  Flpfi^{t^x)  ^  0} 

2.  and  the  initialization  map  :  K  ^  X 

—  U  F^p-^{t,x) 

First,  we  single  out  the  following  property: 

Proposition  32  Let  (F^R)  be  an  impulse  differential  inclusion  defined  on  a 
subset  K.  Knowing  the  substratum  {RjF-)R)f  thus  the  impulse 

map  T^p  and  the  initialization  map  ,  we  can  reconstruct  a  viable  run  of 

the  impulse  differential  inclusion  (F,  R)  through  the  following  algorithm:  Given 
the  cadence  Tn  and  the  initial  state  x^,  we  take 

!i)  the  next  cadence  +1  ^  'F(IP,B)(^n), 
a)  the  next  reinitialized  state  Xn+i  C  F^^ pj(Tn^i,Xn)  C  ^(F,R) 

Hi)  the  next  motive  Xn{')  x{-  Ftn)^  a  solution  to  x'  £  F{x)  satisfying 

a:n(0)  ^  Xn  ^  Xn{Tn+l)  €  R~^{Xn+l) 


(1) 
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In  other  words,  in  terms  of  impulse  times,  given  the  impulse  time  and  the 
initial  state  Xn,  we  take 

'  i)  the  next  impulse  time  tn^i  €  tn  + 

a)  the  next  reinitialized  state  C  U^pj^^{xn) 

\  Hi)  Vt  G  \tn,tn^i],  a  solution  x(*)  to  the  differential  inclusion  x'  G  F{x) 
starting  from  Xn  at  time  tn  viable  in  K  until  it  reaches  R~^{xn^i) 

^  at  time  tn+i* 

(2) 

Proof —  Take  any  run  j;(-)  associated  with  a  sequence  T{x{-))  :=  {t„} 
of  impulse  times  starting  at  xq  e  K  and  viable  in  K,  Then  the  sequence  x  : 
n  x{tn)  is  a  solution  of  the  discrete  dynamical  system  j^-^{tn^i  —  t^Xn), 
obviously  viable  in  K. 

Conversely,  assume  that  the  substratum  r)  is  known.  The  above  algo¬ 
rithm  (2)  starting  at  time  0  and  state  xq  ^  K  provides  a  run  j;(*)  associated 
with  the  sequence  T{x{-))  :=  {tn}  of  impulse  times  of  the  impulse  differential 
inclusion  {F,  R)  viable  in  K.  □ 

Actually,  if  we  are  interested  only  in  the  sequence  of  reinitialized  states  and 
not  necessarily  in  knowledge  of  the  sequence  of  impulse  times,  the  knowledge  of 
the  initialization  map  is  sufficient: 

Proposition  33  A  subset  K  is  viable  under  the  impulse  differential  inclusion 
{F,R)  if  and  only  if  the  domain  of  the  initialization  map  is  equal  to  K. 

Proof  —  Assume  that  K  is  viable  under  {F,  R)  and  prove  that  for  every 
x  ^  K,  ^  0.  Take  any  xq  G  K.  By  definition,  there  exists  a  run  x{-) 

associated  with  a  sequence  T(x(*))  :=  {t„}  of  impulse  times  viable  in  K.  Then 
the  sequence  x  :  n  x[tn)  is  a  solution  of  the  discrete  dynamical  system  , 

obviously  viable  in  K. 

Conversely,  assume  that  K  is  viable  under  the  discrete  system  i.e., 

that  for  every  x  £  K,  U^R-j{x)  ^  0.  We  shall  prove  that  K  is  viable  under  the 
impulse  differential  inclusion  (F,  R) .  Let  xq  given  in  K  and  a  solution  x  :  n  ^ 
Xn  €  U^p^{xn-i)  n  F  to  the  discrete  dynamical  system  By  definition 

of  the  initialization  map  py  we  can  associate  with 

Xn  ^  [J  (^5  ^n-1 ) 


some  Tn-i  £  T^pp-^{xn-i)  such  that 

Xn  :=  XniTn^l)  G  R{d^ {Xn-l,  Xn-l)) 

where  a:„{-)  is  a  solution  to  the  differential  inclusion  x'  £  F{x)  starting  at  time 
0  from  Setting  tn  :=  tn~i-trn-i  and  x{t)  :=  Xn{t-\rtn-i)  if  t  G  [tn-iffn], 
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we  have  checked  that  a;(-)  is  a  run  to  the  impulse  differential  inclusion  (F,  i?) 
associated  with  the  sequence  {tn}n>o  of  impulse  times  tn  starting  from  xq  and 
viable  in  K, 


4  Some  Prerequisite  from  Viability  Theory 

Most  of  the  results  of  viability  theory  are  true  whenever  we  assume  that  the 
dynamics  is  Marchaud: 

Definition  41  (Marchaud  Map)  We  shall  say  that  F  is  a  Marchaud  map  if 

'  i)  the  graph  of  F  is  closed 
ii)  the  values  F(x)  of  F  are  convex 
^  Hi)  the  growth  of  F  is  linear:  3c>0|V2:cX, 

||F(2:)||  :=sup„gj,(^)  I|t)||  <c(||a;||  +  l) 

This  covers  the  case  of  Marchaud  control  systems  where  (a:,  it)  (-)•  /(x,  it)  is 
continuous,  affine  with  respect  to  the  controls  u  and  with  linear  growth  and 
when  P  is  Marchaud. 

We  recall  the  following  version  of  the  important  Theorem  3.5.2  of  Viability 
Theory,  [1,  Aubin]: 

Theorem  42  Assume  that  F  :  X  X  is  Marchaud.  Then  the  solution  map  S 
is  upper  semicompact  with  nonempty  values:  This  means  that  whenever  Xn  ^  X 
converge  to  x  in  X  and  Xn{‘)  C  <5(3^n)  **5  u  solution  to  the  differential  inclusion 
x'  C  F{x)  starting  at  there  exists  a  subsequence  (again  denoted  by)  Xn{’) 
converging  to  a  solution  2:(-)  C  ^(^r)  uniformly  on  compact  intervals. 

Our  purpose  is  to  characterize  the  viability  of  a  subset  K  under  an  impulse 
differential  inclusion: 

Definition  43  We  shall  say  that  a  subset  K  is  viable  under  an  impulse  differ¬ 
ential  inclusion  (F,  P)  if  from  any  initial  state  x  of  K  starts  at  least  one  run 
viable  in  K. 

The  Viability  Theorem^  and  its  consequences  imply  the  following 

Theorem  44  Let  (F,  R)  be  an  impulse  differential  inclusion  and  K  C.  X  be  a 
closed  subset.  Assume  that  F  is  Marchaud  and  that  is  closed.  Then  the 

following  statements  are  equivalent 

1.  K  is  viable  under  (F,  F), 

2.  The  subseP  K\R~^{K)  is  locally  viable  under  F, 

^  See  for  instance  Theorems  3.2.4,  3.3.2  and  3.5.2  of  [1,  Aubin]. 

^  The  subset  K\C  denotes  the  intersection  of  K  and  the  complement  of  C,  i.e.,  is  the 
set  of  elements  of  K  which  do  not  belong  to  C. 
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3.  K,  F  and  R  are  linked  through  the  tangential  condition^ 

VxeK\R-\K),  F(x)  n T/f (x)  0 

(see  [2,  Aubin]  or  [11,  Aubin,  Lygeros,  Quincampoix,  Sastry  &  Seube]  for  a 
proof.) 

We  shall  also  need  some  other  prerequisites  from  Viability  Theory: 

Definition  45  Let  C  C  K  C  X  be  two  subsets,  C  being  regarded  as  a  target, 
K  as  a  constrained  set  The  subset  Capt^(C)  of  initial  states  xq  £  K  such 
that  C  is  reached  in  finite  time  before  possibly  leaving  K  by  at  least  one  solution 
x(-)  G  S{xo)  starting  at  Xq  is  called  the  viable-capture  basin  of  C  in  K.  A  subset 
K  is  a  repelier  under  F  if  all  solutions  starting  from  K  leave  K  in  finite  time. 
A  subset  D  is  locally  backward  invariant  relatively  to  K  if  all  backward  solutions 
starting  from  D  viable  in  K  are  actually  viable  in  K . 

We  shall  use  the  following  characterization  of  capture  basin  (see  [6,  Aubin]): 

Theorem  46  Let  us  assume  that  F  is  Marchaud  and  that  the  subsets  C  C  K 
and  K  are  closed.  If  K\C  is  a  repelier  (this  is  the  case  when  K  itself  is  a 
repelier),  then  the  viable- capture  basin  Capt^(C)  of  the  target  C  under  S  is  the 
unique  closed  subset  satisfying  C  C  D  C  K  and 

{i)  D\C  is  locally  viable  under  S 
ii)  D  is  locally  backward  invariant  relatively  to  K  ^  ' 

5  The  Graph  of  the  Substratum 

We  begin  by  characterizing  the  graph  of  the  substratum  F^j^y, 

Theorem  51  Let  us  assume  that  F  is  Marchaud,  that  C  <Z  R  is  closed  and  that 
the  graph  ofRiC^X  is  closed. 

Then  the  substratum  F^  j^^  :  K  ^  K  is  the  unique  set-valued  map  with 
closed  graph  satisfying 

X  G  K,  F^j^^{0,x)  :=  R{x)  n  K 

and,  for  any  T  >  0 

1.  for  anyy  G  a:),  there  exists  a  solution  a:(-)  to  the  differential  inclu¬ 

sion  x'  G  F{x)  viable  in  K  on  [0,  T]  such  that 

v«  e  [0,r],  y  e  (4) 

The  contingent  cone  Tl{x)  to  L  C  X  sX  x  e  L  is  the  set  of  directions  v  G  X 
such  that  there  exist  sequences  /in  >  0  converging  to  0  and  Vn  converging  to  v 
satisfying  x  +  hnVn  ^  K  for  every  n  (see  for  instance  [8,  Aubin  &  Prankowskaj)  or 
[19,  Rockafellar  &  Wets]  for  more  details). 
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2.  for  any  y  G  K\r^  for  every  solution  x{-)  to  the  differential  inclu¬ 

sion  x'  €  F{x)  viable  in  K  on  [0,  T],  then 

vi€[o,T], 

As  a  consequence^,  for  any  T  >  0  and  for  any  y  G  DkF^ ^^{T,x),  for  every 
solution  x(-)  to  the  differential  inclusion  x'  G  F{x)  satisfying  (4),  then 

V  f  G  [0,  T] ,  2/  G  ^kF^p  (T  —  t,  x{t)) 

For  proving  Theorem  51,  we  shall  first  observe  that  the  graph  of  the  substra¬ 
tum  of  {K,  F,  R)  is  a  viable-capture  basin  and  next,  deduce  the  above  results 
from  the  characterization  of  viable-capture  basins.  Let  us  recall  that  we  denoted 
by  R}^p-  the  graphical  restriction  of  R  to  K  x  K  defined  by 

n  a:  if  X  6 
ifx^K 

We  observe  that  C  :=  Dom(/?j^)  =  K  D  R~^{K),  that  Im{f?j^)  and  that 
Graph(i?|^)  =  Graph(fl)  n  (if  x  K). 

Lemma  52  The  graph  of  the  substratum  of{K,  F,  R)  is  the  viable-capture 

basin  of  {0}  x  Graph(ilj^)  under  the  set-valued  map  {-1}  x  F  x  {O}; 

Graph(r(^_B))  =  Capt^_^jj^^f^oj  ({0}  x  Graph(ilj^)) 

and^  X  €  C  :=  K  n  R-^{K),  x)  =  R{x)  n  K. 

Proof  —  Indeed,  to  say  (T,  x,  y)  belongs  to  the  viable-capture  basin 

({^}  ^  ^raph(J?j^)^ 

means  that  there  exists  a  solution  a:(')  G  5(x)  and  t  G  [0,  T]  such  that 

(i)  Vf€[0,t],  (r-t,x(f),y)  e  Capt^jj^^;^^j,j({0}  X  Graph(ijj^) 
ya)  {T -t,y,x{t))  e  {0}  X  Graph(flj^) 

i.e.,  if  and  only  if  t  =  T  and 

(i)  VtG  [0,T[,  x(t)  G  K 
{  ii)  y  G  RixiT))  n  K 

This  is  equivalent  to  say  that  y  G  F^  (T,  x)  D  K. 

The  relative  boundary  SkD  to  K  of  a.  subset  D  C  K  is  equal  to  jD  O  K\X. 


5 


114  J.-P.  Aubin 


Consequently,  to  say  that  y  belongs  to  x)  means  that  y  e  R{x)  r\K. 

□ 

Proof  of  Theorem  51  —  We  observe  first  that  the  map  {-l}xFx{0}  : 
KxXxX  R  X  X  X  X  is  Marchaud  and  that  R^  x  K  x  K  is  a  repeller  under 
this  map  since  any  solution  {T  —  t,  x{t),y)  starting  at  (T,  x,  y)  leaves  R+  xKxK 
at  time  T.  Theorem  46  states  that  the  viable-capture  basin 

Graph(r(^,«))  =  Captf_Y}fFxV  ({0}  x  Graph(/?|^)) 

is  the  unique  closed  subset  V  C  R  x  A"  x  A"  containing  {0}  x  Graph(Rj^) 
satisfying 

1.  V\({0}  X  Graph(Rj^))  is  locally  viable  under  {—1}  x  F  x  {0} 

2.  and 

Capt^ =  V 

This  states  that  whenever  {T,x,y)  €  (R+  x  K  x  K)\V,  all  solutions  to  the 
differential  inclusion  {t',x',y')  e  {—1}  x  F{x)  x  {0}  leave  (R^.  x  K  x  K) 
before  possibly  reaching  the  target  {0}  x  Graph(Rj^). 

The  first  statement  means  that  whenever  (T,  x,  y)  belongs  to  V,  there  exists 
a  solution  x(-)  to  the  differential  inclusion  x'  G  F{x)  such  that  (T  —  t,x{t),y) 
belongs  to  V  until  it  reaches  {0}  x  Graph(R[^).  This  is  equivalent  to  saying  that 

VtG[0,r],  y&r^%j,^(T-t,x{t)) 

The  second  statement  means  that  whenever  (T,  x,  y)  does  not  belong  to  V,  all 
solutions  x(-)  to  the  differential  inclusion  x'  G  F{x)  are  such  that  {T  ~t,x{t),y) 
do  not  belong  to  V  whenever  (T  -  t,x{t),y)  G  R+  x  K  x  K,  i.e.,  whenever  x(-) 
is  viable  in  K  on  the  interval  [0,T].  This  is  equivalent  to  saying  that  for  all 
solutions  to  x'  G  F{x)  viable  in  K  on  the  interval  [0,T], 

VfG[0,r],  y&K\rl<p^j,^{T~t,x{t)) 

Let  us  consider  now  y  G  dr^j^^{T,x)  where  T  >  0.  This  means  that  there 
exists  a  sequence  yn  ^  K  such  that  y^  G  K\r^  j^^{T,x).  Hence  {T^x^y^)  does 
not  belong  to  the  capture  basin  of  {0}  x  Graph(Rj^)  viable  in  R^.  x  K  x  K. 
Therefore  we  know  that  for  any  solution  x(-)  G  S{x)  viable  in  K  on  [0,T], 
for  any  t  G  [0,r],  y^  G  K\r^j^^{T  —  t,x{t))  and,  in  particular,  that  yn  G 

K\r^  j^J0,x{T))  =  R(x{T)).  Taking  any  solution  x(*)  G  S(x)  satisfying  (4) 
and  the  limit  when  n  +oo,  we  infer  that 

V  t  e  [o,r],  y  e  dKrl^p^R)(T  -  t,x{t)) 


and  that 


y  G  dKR{x{T)) 
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6  Hamilton- Jacobi  Characterization  of  the  Substratum 


Before  stating  the  general  result  characterizing  the  substratum  as  a  solution  to 
a  system  of  first-order  partial  differential  inclusions,  let  us  consider  the  following 
particular  case: 

Proposition  61  Let  us  assume  that  f  :  X  X  is  Lipschitz,  r  :  X  X  is 
single-valued  and  continuous,  that  is  continuous,  that  K  is  viable  under 

{f,r)  and  is  differentiable.  Then  it  is  the  unique  solution  to  the  system 

of  first-order  partial  differential  equations 


yxGK\C,  Vj  =  l,.. 


n, 


duj  {t,  x) 
dt 


E 

i=l 


duj{t,x) 


dxi 


fi{x)  =  0 


or,  in  a  more  compact  form, 

v.sAC,  +  .0 

satisfying  the  condition 


V  a:  €  C,  w(0,a;)  =  r{x) 


We  shall  deduce  from  Theorem  63  below.  Indeed,  thanks  to  the  concepts  of 
contingent  derivative,  we  shall  show  that  the  substratum  unique 

(set-valued)  solution  in  the  “Prankowska  sense”  to  the  “Hamilton-Jacobi  inclu¬ 
sion” 


dV{t,x) 

dt 


dV{t,x) 

dx 


■  F{x) 


(5) 


satisfying  the  condition 


Va:GC,  V(0,x)  =  R{x)f^K 

We  refer  to  [5,7,  Aubin],  [9,  Aubin  &  Frankowska]  and  their  references  for  set¬ 
valued  solutions  to  systems  of  Hamilton-Jacobi  inclusions.  For  that  purpose, 
we  recall  that  the  (graphical  contingent)  derivative  of  a  set-valued  map  V  : 
R+  K  may  be  defined  by  the  relation 

Graph(W(T,a;,3/))  :=  TQj.g^p^^y^{T,x,y) 


Definition  62  We  shall  say  that  a  set-valued  map  V  :  x  K  K  is  a 

Frankowska  solution  to  the  Hamilton-Jacobi  system  of  first- order  partial  differen¬ 
tial  inclusions  (5)  satisfying  the  initial  condition  y(0,x)  =  R{x)  if  its  graph  is 
closed,  if 


W  t  >  0,  y  y  £  V{t,x),  3  V  £  such  that  0  £  DV{t,x,y){—l,v) 
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and  if  for  every  v  G  F(x) 

V  i  >  0,  V  2/  €  V{t,x),  0  G  DV{t,x,y)(l,  -v) 
or 

{i)  —vG  Tx\k(^)  if  ^  ^  dK 
\  ii)  ~v  G  Tk{x)  if  2/  G  dK 

Theorem  63  Let  us  assume  that  F  is  Marchaud,  that  C  :=  K  n  R~^(K)  is 
closed  and  that  the  graph  of  R  :  C  ^  K  is  closed. 

1.  The  substratum  F^  :  K  ^  K  is  the  largest  set-valued  map  V  :  H^xK 
K  with  closed  graph  contained  in  K  x  K  satisfying 

\/  t  >  0,  y  E  V{t,x),  3  t;  G  F{x)  such  that  0  G  DV{t,x.,y)(—l^v) 

and  the  condition  1^(0,  x)  —  R{x)  n  Ky 

2.  If  furthermore,  F  is  assumed  to  be  Lipschitz,  the  substratum  F^p  :  K  K 
is  the  unique  Frankowska  solution  V  :  R_|_  xK  ^  K  to  the  Hamilton- Jacobi 
system  of  first-order  differential  inclusions  (5)  satisfying  the  initial  condition 
y(0,rr)  =R(x). 

Proof  —  When  F  is  Marchaud,  to  say  that  Graph (V^)\({0}  x 
Graph  is  locally  viable  under  {-1}  x  F  x  {0}  means  that 

V  {t,x,y)  €  Graph(V)\({0}  x  Graph  (i?|^))  , 

{“!}  'x  F{x)  X  {0}  ^Graph(V)(^’ 

We  observe  that  {t,x,y)  G  Graph(V)\({0}  x  Graph{i2j^))  whenever  t  >  0  and 
we  recall  that 

^Graph(V)(^’^’2/)  =  Gra,ph(BV(t,x,y)) 
so  that  the  above  condition  reads 

V  t  >  0,V  y  €  r^^P^(t,x),  3  V  e  F(x)  such  that  0  G  BV(t,x,y)(—l,v) 
When  F  is  assumed  to  be  Lipschitz,  to  say  that 

Captf_Yj^;,'^^o}(Graph(V))  =  Graph(V) 

means  that 

1.  for  any  (t,x,y)  G  Graph(l^))  n  Int(R-f-  x  K  x  K), 

({1}  X -F(x)  X  {0})  C  =  Graph(DF(«,x,j/)) 

This  is  equivalent  to  say  that  for  every  v  G  F(x), 

V  t  >  0,  X  E  Int(K),  y  G  V(t,x)  nlnt(K),  0  G  BV(t,x,y)(l, —v)  (6) 
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2.  and  otherwise,  for  any  {t,x,y)  G  Graph(y))  Pi  ^(R-i-  x  K  x  K), 

({1}  X -F(a;)  X  {0})  c  U  T(RxA:xX)\(R+xKxK)(i>a;,t/) 

This  means  that  for  every  v  G  F{x), 

{z)  0  G  DV{t,x,y){l,~v)  lit  =  O^y  e  R(x) 

ii)  0  G  D{t^  X,  2/)(l,  —v)  or  —  V  £  Tx\k  if  t  >  0,  x  G  dK,  y  G  R{x) 

Hi)  0  G  D{t,  X,  ?y)(l,  —v)  or  —  t;  G  if  t  >  0,  ye  R{x)  n  dK 

Indeed, 

(R  X  X  X  X)\(R+  xKxK)  = 

(R_  X  X  X  X)  U  (R+  X  {X\K)  X  X)  U  (R+  X  X  X  {X\K)) 

Therefore,  condition  (1,  — u, 0)  belongs  to  the  contingent  cone  to  R_  xKxK 
at  (0,x,  2/)  is  impossible,  condition  (1,  —v,0)  belongs  to  the  contingent  cone 
to  R_  X  {X\K)  X  K  at  (t,  x,  i/)  when  x  G  dK  means  that  —v  belongs  to 
Tx\k{^)  and  condition  (1,— u,0)  belongs  to  the  contingent  cone  to  R_  x 
K  X  {X\K)  at  {t,x,y)  when  y  G  dK  means  that  —v  belongs  to  Tk(x).  □ 

For  the  initialization  map,  we  obtain  the  following  Hamilton- Jacobi  inclu¬ 
sion  : 

Theorem  64  Let  us  assume  that  F  is  Marchaud,  that  C  K  f)  R~^{K)  is 
closed  and  that  the  graph  of  R  :  C  ^  K  is  closed. 

1.  The  initialization  map  :  K  ^  K  is  the  largest  set-valued  map  V  : 

R-|.  X  K  K  with  closea  graph  contained  in  K  x  K  satisfying 

V  3/  G  V(x),  3ve  F{x)  such  that  0  G  DV{Xjy){v) 

2.  If  furthermore,  F  is  assumed  to  be  Lipschitz,  the  initialization  map  : 

K  K  is  the  unique  Frankowska  solution  V  :  R_|.  x  K  ^  K  to  the 
Hamilton- Jacobi  system  of  first-order  differential  inclusions  (5)  satisfying 
the  condition  V  x  G  C,  V(x)  =  R{x). 
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Abstract.  Path-dependent  impulse  differential  inclusions,  and  in  partic¬ 
ular,  path-dependent  hybrid  control  systems,  are  defined  by  a  path- 
dependent  differential  inclusion  (or  path-dependent  control  system,  or 
differential  inclusion  and  control  systems  with  memory)  and  a  path- 
dependent  reset  map. 

In  this  paper,  we  characterize  the  viability  property  of  a  closed  subset  of 
paths  under  an  impulse  path- dependent  differential  inclusion  using  the 
Viability  Theorems  for  path- dependent  differential  inclusions. 

Actually,  one  of  the  characterizations  of  the  Characterization  Theorem 
is  valid  for  any  general  Impulse  evolutionary  system  that  we  shall  defined 
in  this  paper. 

Keywords;  hybrid  control,  impulse  control,  path-dependent  differential 
inclusion,  differential  inclusion  with  memory,  functional  differential  in¬ 
clusions,  viability,  run,  execution,  Kakutani  Theorem,  contingent  cone, 
Marchaud  map. 


Introduction 

In  this  paper,  we  characterize  the  viability  property  of  a  closed  subset  of  paths 
under  an  impulse  path-dependent  differential  inclusion  using  the  method  of 
[2,  Aubin]  or  [8,  Aubin,  Lygeros,  Quincampoix,  Sastry  &  Seube],  the  Path- 
Dependent  Viability  Theorems  of  [12,13,14,  Haddad]. 

Actually,  one  of  the  characterizations  of  the  Characterization  Theorem  is  true 
for  any  general  impulse  evolutionary  system  that  we  shall  define  in  this  paper, 
which  is  based  on  recent  results  of  [5,  Aubin]. 

We  recall  that  hybrid  control  systems^  can  be  embedded  in  the  framework 
of  impulse  differential  inclusions;  in  the  same  way,  path-dependent  hybrid  sys¬ 
tems  can  be  regarded  as  instances  of  viable  path-dependent  impulse  differential 
inclusions,  and  enjoy  the  same  properties. 

^  See  for  instance  among  many  papers  and  books  [10,  Branicky,  Borkar  &  Mitter],  [9, 
Bensoussan  &  Menaldi],  [15,16,  Matveev  &  Savkin]  and  [18,  Shaft  &  Schumacher]. 


M.D.  Di  Benedetto,  A.  Sangiovanni-Vincentelli  (Eds.):  HSCC  2001,  LNCS  2034,  pp.  119-132,  2001. 
(c)  Springer- Verlag  Berlin  Heidelberg  2001 
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1  Impulse  Path-Dependent  Differential  Inclusions 

Let  X  :=  be  a  finite  dimensional  vector  space. 

1.1  History  Spaces 

We  denote  by 

niX)  :=  C(-oo,0;X) 
the  history  (or  memory,  path)  space  . 

It  is  supplied  with  the  compact  convergence  topology.  We  denote  by  'H\{X) 
the  subset  of  Lipschitz  functions  with  Lipschitz  constant  A. 

If  K  C  7{(X),  we  set 

VTe]-oo,0],  K(t)  :=  {vW}^pgK 

Observe  that  Ascoli’s  Theorem  states  that  a  closed  subset  K  C  'Hx(X)  is 
compact  if  and  only  if  K(0)  :=  {v?(0)}(^gK  is  bounded^  since  it  is  closed  and 
equicontinuous  (by  assumption)  and  pointwise  bounded  because,  for  all  €  K 
and  r  <  0, 


II^WII  <  Mr)  -  ^(0)11  +  ||V-{0)||  <  A|r|  +  ||K(0)|| 

Our  study  invlves  a  constrained  subset  K  C  ftiX)  made  of  paths  or  histories 
and  of  a  target  C  C  K. 

A  first  example  of  constrained  subset  of  paths  or  histories  and  targets  asso¬ 
ciated  with  subsets  C  C  K  C  X  oi  the  vector  space  are  given  by 

C  :=  {ip  G  n{X)  I  <p{0)  eC}cK:={ipe  n{X)  I  <p{0)  eK} 

where  the  constraints  bear  only  on  the  present. 

Another  class  is  given  by  Volterra  sets  defined  through  a  “kernel”  k  :]  —  oo,  0]  x 
X  Y  and  a  set- valued  map  M  :Y  X  hy 

K  ;=  l^^p&'H{X)\ip{Q)  e  k{-s,v{s))dn{s) 

where  the  constraints  involve  cumulated  consequences  of  the  history. 

In  the  discrete  case, 


K  :=  \  pGn{X)\p^{0)  €  M  ^ 


J=~oo 


involves  discrete  cumulated  consequences  of  the  history  (delays). 

Associated  targets  can  be  asociated  with  set- valued  maps  P  C  M  in  the  same 
fashion. 
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1.2  Histories  or  Paths 

We  associate  with  any  continuous  function  a:(-)  G  C(— oo,4-oo;X)its  history  (or 
pathp  T{t)x  up  to  time  t  defined  by: 

Vtg]  — oo,0],  r(i)x(r)  :=  x{t-\-r) 

Then  T{t)  maps  C(— oo,+oo;X)  to  HiX)  and  satisfies  the  semi-group  property 

T{t-\-s)x  =  T{s)T(t)x 

We  then  observe  that  for  any  function  x{-)  G  C(— oo, +oo;  X),  we  have  a:(t)  = 
(T(t)x)(0). 

In  this  continuous  framework,  we  define  the  constraints  of  the  history  of  the 
evolution  through  a  closed  subset  K  C  H{X).  Viable  evolutions  x(‘)  with 
memory  are  the  ones  that  satisfy 

Vt>0,  T(t)x  G  K  (1) 


and  an  evolution  x(-)  reach  a  target  C  at  time  sifT(s)xGC. 

For  instance,  an  evolution  is  viable  in  K  :=  {(^  G  'H(X)  |  (^(0)  G  K}  if  and 
only  if  for  every  i  g]  —  oo,  0],  x{t)  G  K.  If 


K,{ 


if  G  n{X)  I  cp{0)  G  M 


(£ 


k{-s, 


)} 


then  2:(-)  is  viable  in  K  if  and  only  if 


V  t  >  0,  x{t)  G  M 


(/ 


k{t  —  s,  (p{s))dfj,{s) 


1.3  Path-Dependent  Differential  Inclusions 

Let  us  consider  a  set- valued  map  F  :  'H{X)  X  governing  the  continuous 
evolution  of  the  state  x{t)  through  the  path-dependent  differential  inclusion 

for  almost  all  i  >  0,  x'(t)  G  F{T{t)x) 

starting  at  a  given  (p  G  V-iX)  in  the  sense  that 

T(0)x  =  (f 

i.e.,  for  every  r  g]  —  oo,0],  x{t)  = 

We  denote  by  Tip  '  '^{X)  ^  C(0,  oo;  X)  the  map  associating  with  any  initial 
path  (p  G  'H{X)  the  set  'R,p{<p)  of  solutions  t  x{t)  to  the  path-dependent 
differential  inclusion  x'{t)  G  F{T{t)x)  starting  at  the  initial  path  (p  in  the  sense 
that  T{0)x  =  ip. 


^  often  denoted  by  :=  T{t)x 
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Actually,  we  shall  need  the  properties  of  the  associated  set-valued  map  Sp  : 
n{X)  C{0,oo-,n{X))  defined  by 

Spiip)  :=  {t  (^>(0 


Definition  11  We  shall  say  that  this  set-valued  map  Sp  ‘  'hi{X)  ^  C(0,  -foo  ; 
'H.{X))  is  the  solution  map  of  F, 

The  solution  map  Sp  has  the  advantage  of  mapping  the  set  'H{X)  into  time- 
dependent  functions  t  (p{t)  :=  T{t)x  that  belong  to  the  same  histoy  space 
7i{X),  even  though  the  traditional  view  is  to  call  a  solution  a  function  t  ^  x{t), 
taking  its  values  in  X. 

This  choice  oi  Sp  instead  oiltp  is  justified  by  its  following  properties: 

1.  the  translation  property:  Let  <^(')  G  S[<p).  Then  for  all  s  >  0,  the  function  V^(-) 
defined  by  V'CO  :=  ^{t  +  s)  is  the  history  :=  T(*)y  G  S{T{s)x)  of  the 
solution  y{-)  to  the  path-dependent  differential  inclusion  starting  at  T{s)x, 

2.  the  concatenation  property:  Let  (/?(•)  €  Sp{(p)  be  the  history  of  a  solution  to 
the  path-dependent  differential  inclusion  starting  at  the  path  and  s  >  0. 
Then  for  every  history  G  Sp{T{s)x)  of  a  solution  y[')  to  the  path- 
dependent  differential  inclusion  starting  at  the  initial  path  T{s)x^  the  func¬ 
tion  ^(•)  defined  by 


f  (p(t)  :=  T{t)x  if  t  G  [0,  s] 

\  'iplt  —  s)  :=  T{t  —  s)y  ift  >  s 


is  the  history  of  the  solution  z{’)  defined  by 


z{t) 


(  x{t)  if  t  G  [0,  s] 
I  —  T)  if  t  >  s 


to  the  path-dependent  differential  inclusion  starting  at  the  initial  path  y?, 
and  thus,  belongs  to  Sp{ip). 

These  two  properties  to  which  we  add  the  upper  compactness  of  the  solution 
map  are  enough  to  obtain  relevant  (and  interesting)  properties  of  path-dependent 
impulse  differential  inclusions,  common  to  other  dynamical  systems. 


1.4  Runs  of  Impulse  Path-Dependent  Differential  Inclusions 

We  now  introduce  a  constrained  functional  set  K  C  'H{X)^  a  functional  target 
C  C  K  and  a  path-dependent  reset  map  R  :  C  ^  K.  with  nonempty  values 
R{ip). 

The  pair  {F,  R)  governs  the  evolution  of  impulse  systems  in  the  following 


sense. 
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Definition  12  Let  us  consider  a  finite  dimensional  vector  space  X ,  the  space 
7{{X)  of  histories,  a  subset  K  C  'H{X),  a  target  C  C  K,  a  set-valued  map 
F:n{X)  X  and  a  set-valued  map  R  :  C  Ti  with  nonempty  values,  regarded 
as  a  path-dependent  reset  map.  We  regard  the  pair  {F,R)  as  the  dynamics  of  a 
path-dependent  impulse  differential  inclusion. 

A  run  of  the  path- dependent  impulse  differential  inclusion  {F,  R)  is  defined 
by 

1.  a  finite  or  infinite  sequence  T(a:(-))  :=  {t„}„  of  nonnegative  cadences  € 
[0,oo[, 

2.  a  sequence  of  reinitialized  paths  ifn  G  'H(X), 

3.  a  sequence  of  motives  (pn{-)  •=  F{-)xn  ^  where  (pn  G  RiX)  is  the 

history  of  a  solution  a:n(-)  to  the  path- dependent  differential  inclusion  x'{t)  G 
F{T(t)x)  starting  at  the  initial  path  pn  and  satisfying  the  end-point  condition 

T(Tn)Xn  e 

by 

{i)  defining  the  sequence  of  impulse  times  tn+i  :=  tn+Tn, 

ii)\/ t  e[tn,tn+ll  x{t)  :=  Xn{t-tr^) 

If  the  sequence  of  cadences  is  finite^  and  stops  at  Tj\f,  we  agree  that  the  Nth 
motive  is  defined  on  [0,  -}-oo[,  i.e.,  that  we  take  rjv+i  =  +oo. 

We  say  that  a  run  x{’)  is  viable  in  K  if  for  any  t  >  0,  T{t)x  G  K  and  that 
K  is  locally  viable  under  (F,  R)  if  for  any  G  K,  there  exists  at  least  one  run 
of  the  impulse  path- dependent  differential  inclusion  viable  on  a  nonempty  time 
interval  and  (globally)  viable  if  it  is  viable  on  [0, -f-oo[. 

At  this  stage,  a  run  x{’)  can  just  be  a  (discrete)  sequence  of  paths  (pn+i  ^ 
R{(pri)  at  the  initial  time  (case  when  for  all  n  >  0,  the  cadences  =  0),  or  just 
a  (continuous)  solution  x(-)  to  the  path-dependent  differential  inclusion  x'(t)  G 
F{T{t)x)  (case  when  Tj  =  +oo),  or  an  hybrid  of  these  two  path-dependent 
modes,  the  discrete  and  the  continuous. 

Path-dependent  hybrid  systems  can  be  regarded  as  instances  of  viable  path- 
dependent  impulse  differential  inclusions  as  in  the  case  of  usual  hybrid  systems: 
we  refer  to  [2,  Aubin]  or  [8,  Aubin,  Lygeros,  Quincampoix,  Sastry  &  Seube]  for 
more  details  on  this  topic. 


2  Statement  of  the  Impulse  Path-Dependent  Viability 
Theorem 

2.1  Marchaud  Maps 

The  Viability  Theorems  hold  true  whenever  we  assume  that  the  dynamics  gov¬ 
erning  the  path-dependent  evolution  is  Marchaud: 


^  We  shall  see  that  we  can  eliminate  this  situation  by  assuming  that  R(C)  fl 
ViabF(K)  =  0. 
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Definition  21  (Marchaud  Map)  We  shall  say  that  F  :  ^  X  is  a  Mar¬ 

chaud  map  if 


ii)  F  is  upper  semicontinuous 
ii)  the  values  F{<p)  of  F  are  convex 
Hi)  the  growth  of  F  is  linear:  3  c>  0  |  V  €  'H(X), 

||F(¥p)||  :=  sup„gp,(^,  ||t)||  <  c(||v3(0)||  +  1) 

This  covers  the  case  of  Marchaud  control  systems  where  w)  /(</?,  w)  is 
continuous,  affine  with  respect  to  the  controls  u  and  with  linear  growth  and 
when  P  :  'H(X)  Z  is  Marchaud. 

We  recall  the  following  version  of  the  important  Haddad  Theorem  12.4.1  of 
[1,  Aubin]: 

Theorem  22  Assume  that  F  :  l-i{X)  X  is  Marchaud.  Then  its  solution 
map  Sp  is  upper  semicompact  with  nonempty  values:  This  means  that  whenever 
G  'H(A')  converge  uniformly  on  compact  intervals  to  in  'W(X)  and  any 
history  (pn{')  '=  T{-)xn  G  Sp{(pn)  associated  to  a  solution  Xn{-)  to  the  path- 
dependent  differential  inclusion  F{t)  G  F{T{t)x)  starting  at  there  exists  a 
subsequence  (again  denoted  by)  (pn{‘)  converging  uniformly  on  compact  intervals 
to  the  history  (/?(•)  :=  T{')x  of  a  solution  a:(-)  to  the  path- dependent  differential 
inclusion  starting  at  (p. 


2.2  Contingent  Directions 

In  the  case  of  path-dependent  impulse  differential  inclusions,  we  shall  charac¬ 
terize  the  viability  of  a  functional  constrained  set  K  in  terms  of  contingent 
directions  to  a  K  C  ^[{X)  be  a  subset  of  histories  at  a  path  ip  G  Ti.  Let 

A{X)  :=  {x(-)  G  C(0, -f  oo;  X)  such  that  rr(0)  =  0} 

denote  the  “future  space”.  We  embed  the  state  space  X  into  A{X)  by  identifying 
a  vector  x  with  the  function  :=  tx.  The  image  of  the  ball  XBx  of  radius  A 
under  this  embedding  is  contained  in  of  A-Lipschitz  functions. 


Definition  23  Let  h>0  be  given.  The  ^-concatenation  for  concatenation  when 
there  are  no  ambiguities)  ipOhi^  the  bilinear  form  from  ^{X)  x  A{X)  i->  'H(X) 
defined  by 


f  p{t  -h  /i)  if  r  e]  —  oo,  -h\ 

\  (p{0)  -I-  'ilj(r  -b  h)  if  T  G  [-/i,  0] 


As  an  example,  the  concatenation  of  (/?  G  ^(X)  and  x  G  X  is  defined  by 


{(pOhX)(T) 


(  p{t  -\-h)  if  r  g]  -  oo,  h] 

\  (p(0)  +  (r  -j-  h)x  if  r  G  [-/i,  0] 
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We  attach  to  a  function  ip  £  'H{X)  and  a  sequence  vi,...,Vn  the  Euler 
concatenation  defined  by 


(pOhVlOhy20h'-OhVn 

which  is  piecewise  linear  function  on  the  interval  [— n/i,  0]. 

Lemma  24  For  any  Lipschitz  constant  A  >  0,  the  h- concatenation  maps 
nx{x)xAxiX)  to  nx{x) 


Definition  25  We  denote  by  the  set  of  vectors  v  £  X  such  that  there 

exist  a  sequence  h^  >  0  converging  to  0  and  a  sequence  Vn  £  X  converging  to  v 
satisfying 

V  n  >  0,  ipOh^Vn  e  K 

2.3  The  Impulse  Path-Dependent  Viability  Theorem 

Theorem  26  Let  (F,  R)  be  a  path- dependent  impulse  differential  inclusion  and 
K  C  'H{X)  be  a  closed  subset.  Assume  that  F  is  Marchaud  and  that  C  G  K.  is 
closed.  Then  the  following  statements  are  equivalent 

1.  K  is  viable  under  (F,R), 

2.  The  subset  K\C  is  locally  viable  under  the  path- dependent  differential  inclu¬ 
sion  governed  by  F, 

3.  K;  C;  jP  and  R  are  linked  through  the  tangential  condition 

VvpeK\C,  F{ip)nVK{T)j^^ 

Actually,  both  impulse  differential  inclusions  and  path-dependent  impulse 
differential  inclusions'^  share  the  same  properties  at  a  higher  abstraction  level, 
the  level  of  impulse  evolutionary  systems  we  are  about  to  define.  It  is  at  this 
level  that  the  two  first  statements  are  equivalent. 

The  equivalence  between  the  second  and  third  statement  is  specific,  and 
provided  in  our  case  by  the  Path- Dependent  Viability  Theorems  of  [12,13,14, 
Haddad]. 


2.4  Examples 

Take  any  path-dependent  differential  inclusion  x'{t)  £  F{T{t)x)  associated  with 
a  Marchaud  right-hand  side  F. 

We  refer  to  Chapter  12  of  [1,  Aubin]  for  examples  of  tangential  conditions 
when  the  constrained  set  K  and  the  constrained  targets  C  C  K  are  Volterra 
sets  defined  by  kernels,  by  lack  of  space.  Consider  only  the  simple  case  when 


as  well  as  parabolic  (or  reaction-diffusion  type)  partial  differential  inclusions  and 
mutational  equations  governing  the  evolution  of  subset. 
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the  constrained  subset  of  paths  or  histories  and  the  target  are  associated  with 
subsets  C  (Z  K  C  X  oi  the  vector  space  X  by 

C  G  li{X)  \  ifiO)  eC}  CK  :={ipe  n{X)  \  ip{0)  G  K  } 

Assume  that  the  reset  map  R  is  associated  with  an  usual  reset  map  Rq  :  C 
K,  where  C  c  K  hy  the  formula 

V(^G?^(X),  VrG]-oo,0],  (i?(^))(r)  :=  Ro{ip{r)) 

In  this  case,  a  run  of  the  path-dependent  impulse  differential  inclusion  (F,  Rq) 
is  defined  by 

1.  a  finite  or  infinite  sequence  t(2:(-))  :=  {Tn}n  of  nonnegative  cadences  G 
[0,oo[, 

2.  a  sequence  of  reinitialized  states  Xn  £ 

3.  a  sequence  of  motives  that  are  solutions  to  the  path-dependent  differen¬ 
tial  inclusion  x'(t)  G  F{T{t)x)  starting  at  the  initial  state  Xn  and  satisfying 
the  end-point  condition  £  R{xn{Tn)) 

by 

{i)  defining  the  sequence  of  impulse  times  :=  tn  + 
ii)y  t  e[tn,tn+l[,  x{t)  ^  ^ 

Theorem  27  Let  F  be  a  path- dependent  Marchaud  set-valued  map,  K  and  C  C 
K  and  Rq  \  C  K  he  a  reset  map.  Then  K  is  viable  under  {F,  Rq)  if  and  only 
if  the  tangential  condition 

y  ip  £  %[X)  such  that  (p{0)  £  K\C,  F{<p)  D  Tk{<p{^))  ^  0 


2.5  Path-Dependent  Hybrid  Systems 

Definition  28  An  path- dependent  hybrid  differential  inclusion  {K,  F,  Rq)  is  de¬ 
fined  by 

1.  a  finite  dimensional  vector  space  E  of  states  e  called  locations, 

2.  a  set-valued  map  K  :  E  X  associating  with  any  location  e  a  (possibly 
empty)  subset  K{e)  C  X  and  a  set-valued  map  C  :  E  X  associating  with 
any  location  e  a  (possibly  empty)  subset  C{e)  C  K{e), 

3.  a  set-valued  map  F  :  E  x  1-L{X)  ^  X  with  which  we  associate  the  path- 
dependent  differential  inclusion  x'(t)  £  F{e,T(t)x), 

4.  a  set-valued  map  (reset  map)  Rq  :  Graph  (C)  Graph  (AT). 

A  run  of  the  path- dependent  hybrid  system  is  defined  by 

1.  a  finite  or  infinite  sequence  r(e,a:(-))  :=  {r^jn  of  nonnegative  cadences  £ 
[0,oo[, 

2.  a  sequence  of  locations  and  of  reinitialized  states  Xn  £  K{en), 
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3.  a  sequence  of  motives  Xn{’)  that  are  solutions  to  the  path- dependent  dif¬ 
ferential  inclusion  x\t)  G  F{en^T{t)x)  starting  at  the  initial  state  Xn  and 
satisfying  the  end-point  condition  Xn+i  G  Ro{^n,^n{Tn)) 

by  (3). 

We  observe  right  away  that  a  map  (e,a;(-))  is  a  run  of  the  hybrid  differential 
inclusions  if  and  only  if  (e(-),x(*))  is  a  run  of 

r  i)  e'{t)  =  0 

G  F{e{t),T(t)x) 

"viable”  In  Graph(iir)  until  it  reaches  the  graph  of  the  map  C.  Indeed  the  loca¬ 
tions  remain  constant  in  the  intervals  since  their  velocities  are  equal  to 

0. 

Since  the  existence  of  solutions  to  path-dependent  hybrid  differential  inclu¬ 
sions  amounts  to  the  viability  of  the  graph  of  the  set-valued  map  K  under  an 
associated  auxiliary  path-dependent  impulse  differential  inclusion,  we  obtain  a 
necessary  and  condition  for  the  existence  of  solutions  to  hybrid  differential  in¬ 
clusions  thanks  to  Theorem  27.  For  that  purpose,  we  need  the  definition  of  the 
contingent  derivative  DK{e,x)  :  E  X  oi  a.  set- valued  map  K  :  E  X  at  a, 
point  (e,  x)  of  its  graph:  It  can  be  defined  by 

Graph(DA:(e,a:))  :=  7Gj,aph(A:)(®>®) 


Theorem  29  Let  {K,F,Ro)  be  a  path- dependent  hybrid  differential  inclusion. 
Assume  that  F  is  Marchaud.  Then  the  path- dependent  hybrid  differential  inclu¬ 
sion  has  a  solution  for  every  initial  state  if  and  only  if 

W  e  G  E,W  cp  e  TLiX)  such  that 

q>(0)GK{e)\K{e)\C{e),  F{e,ip)  D  DK{e,piO)m  ^  iji 

3  Impulse  Evolutionary  Systems 

Therefore,  it  costs  nothing  to  prove  the  equivalence  between  the  two  first  state¬ 
ments  in  the  general  case  of  impulse  evolutionary  systems: 

3.1  Impulse  Evolutionary  Systems 

Definition  31  An  evolutionary  system  is  a  set-valued  map  S  :  X  C(0,  oo;  X) 
satisfying 

1.  the  translation  property:  Let  x(-)  G  <S(x).  Then  for  all  T  >  0,  the  function 
y{-)  defined  by  y{t)  :=  x{t-{-T)  is  a  solution  y{-)  G  S{x{T))  starting  at  x{T), 
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2.  the  concatenation  property;  Let  a:(-)  G  5(x)  and  T  >  0.  Then  for  every 
y{')  G  S{x(T)),  the  function  z(-)  defined  by 


x{t)  iftG[0,T] 
yit-T)  ifi  >T 


belongs  to  S{x). 

We  can  define  impulse  evolutionary  systems  in  the  following  way: 

Definition  32  Let  K  C  X ,  C  C  K  be  two  nonempty  subsets  and  R  :  C  K 
a  set-valued  map^  with  nonempty  values,  regarded  as  a  reset  map,  and  S  :  X  ^ 
C(0,  oo;  X)  be  an  evolutionary  system.  Then  the  pair  (<S,  R)  governs  a  run  x(‘) 
of  an  impulse  evolutionary  system  defined  by 

1.  a  finite  or  infinite  sequence  r{x{>))  :=  {Tn}n  of  nonnegative  cadences  G 
[0,  +oo[, 

2.  a  sequence  of  reinitialized  states  Xn, 

3.  a  sequence  of  motives  Xn{‘)  G  S{xn)  satisfying  the  end-point  condition 
^niTri)  €  R  ^(^n+l) 

by 

{i)  defining  the  sequence  of  impulse  times  ‘=tn+Tn, 

zi)  V  t  G  [tnj  tn+l  [i  —  ^n) 

If  the  sequence  of  cadences  is  finite^  and  stops  at  rj^f,  we  agree  that  the  Nth 
motive  xn{-)  G  S{xn)  is  taken  on  [0,  +oo[,  i.e.,  and  we  agree  to  set  =  +oo. 

We  say  that  a  run  x{-)  is  viable  in  K  if  for  any  t  >0,  x{t)  G  K  and  that 
a  closed  subset  K  is  viable  under  an  impulse  evolutionary  system  {S,R)  if  from 
any  x  £  K  starts  at  (east  one  run  viable  in  K. 

In  order  to  characterize  the  viability  of  K  under  an  evolutionary  system,  we 
also  need  the  following  definitions: 

Definition  33  Let  S  :  X  C(0,-foo;X)  be  a  set-valued  evolutionary  system 
and  K  C  X  be  a  subset  regarded  as  a  constrained  set 

The  subset  K  is  said  locally  viable  under  S  if  from  any  initial  state  x  ^  K 
starts  at  least  one  solution  viable  in  K  on  a  nonempty  interval  and  viable  if  this 
solution  is  viable  on  [0,  +oo[. 

The  viability  kernel  Viab{Ar)  is  the  subset  of  initial  states  xq  e  K  such  that 
one  solution  a:(-)  G  <S(xo)  starting  at  xq  is  viable  in  K  for  all  t>0.  A  subset  K 
is  a  repeller  under  S  if  its  viability  kernel  is  empty. 

^  When  R  :  X  X  is  defined  on  X,  we  associate  with  it  its  “graphical  restriction” 
to  K  X  K  (again  denoted  by)  R  where  C  :=  X  Pi  R~^{K)  and  R{x)  is  replaced  by 
R{x)  n  K. 

®  We  shall  see  that  we  can  eliminate  this  situation  by  assuming  that  R(C)  n 
Viabs(X)  =  0. 


Path-Dependent  Impulse  and  Hybrid  Systems  129 

3.2  Characterization  of  Impulse  Evolutionary  Systems  Viability 

We  can  adapt  the  proof  of  [2,  Aubin]  or  [8,  Aubin,  Lygeros,  Quincampoix,  Sastry 
h  Seube]  for  characterizing  the  viability  of  a  closed  subset  under  an  impulse 
evolutionary  system: 

Theorem  34  Let  {S^R)  he  an  impulse  evolutionary  system  and  K  C  X  and 
C  <Z  K  be  closed  subsets.  Assume  that  S  is  upper  semicompact.  Then  the  follow- 
ing  statements  are  equivalent 

1.  K  is  viable  under 

2.  The  subset  K\C  is  locally  viable  under  S, 


3.3  Prerequisites  of  Viability  Theory 

For  proving  this  characterization  theorem,  we  need  some  results  of  viability 
theory. 


Definition  35  Let  K  C  X  be  a  subset.  The  functional  tk  •  C(0,  oo;X) 

R-|.  U  {+oo}  associating  with  x{-)  its  exit  time  defined  by 

rK{^{-))  ■■=  inf  {i  e  [0,oo[  I  x{t)  i  K}  := 

is  called  the  exit  functional. 

Let  C  C  K  be  a  target.  We  introduce  the  (constrained)  hitting  functional 
'^iK,c)  defined  by 

•=  inf{t  >  0  I  x{t)  €  C  &:  Vs  e  [0,  t],  x(s)  £  K  } 

associating  with  x{-)  its  hitting  time,  introduced  in  [11,  Cardaliaguet,  Quincam¬ 
poix  &  Saint-Pierre]). 

Consider  an  evolutionary  system  S  \  X  ^  C(0, -foo;  A).  Let  C  c  K  and  K 
be  two  subsets. 

The  function  R+  U  {+oo}  defined  by 

x{)GS{x) 


is  called  the  upper  exit  function. 

The  function  :  AT  R+  U  {+oo}  defined  by 


inf 

xi-)es{x) 


is  called  the  lower  constrained  hitting  function. 


We  shall  need  the  following 
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Theorem  36  Let  S  :  X  C(0,+cx:);X)  be  a  strict  upper  semicompact  map 
and  C  and  K  be  two  closed  subsets  such  that  C  C  K.  Then  the  hitting  function 
^\k,C)  lower  semicontinuous  and  the  exit  function  is  upper  semicontinu- 
ous.  Furthermore,  for  any  x  €  Dom('C[7^^  there  exists  at  least  one  solution 
€  ^{x)  which  hits  C  as  soon  as  possible  before  possibly  leaving  K 

and  for  any  x  e  Dom(r|(-),  there  exists  at  least  one  solution  e  S{x)  which 
remains  viable  in  K  as  long  as  possible: 

xUx)  =  TKix'^i-)) 

(See  [5,  Aubin]  for  a  proof  and  more  details  on  evolutionary  systems). 

3.4  Proof  of  the  Characterization  of  Impulse  Evolutionary  Systems 
Viability 

Indeed,  if  K  is  viable  under  {S,R),  then  from  any  xq  £  K\C  starts  at  least  a 
solution  a:(-)  £  ^(x)  viable  in  K 

1.  either  forever  if  xq  belongs  to  the  viability  kernel  Viab(A')  of  K 

2.  or  until  it  reaches  at  some  time  ti  >  0  a  state  x(“ti)  in  C. 

This  shows  that  K\C  is  locally  viable. 

Conversely,  let  us  assume  that  K\C  is  locally  viable  and  take  an  initial  state 
xo  G  K.  If  xo  belongs  to  C,  we  may  take  tq  =  0  and  xi  £  jR(xo).  Consider  now 
the  case  when  xq  £  K\C. 

If  Xo  belongs  to  Viab(i^),  then  at  least  one  solution  starting  from  xq  is  viable 
in  K,  and  thus,  defines  a  run  viable  in  K:  We  may  take  the  cadence  tq  =  +00 
and  for  motive  a  solution  xo(-)  £  S{xo)- 

If  Xo  does  not  belong  to  Viab(Ar),  all  solutions  leave  K  in  finite  time  before 
(possibly)  reaching  the  viability  kernel.  It  is  then  enough  to  prove  that  at  least 
one  of  them  reaches  C  before  leaving  K.  This  is  the  case  of  a  solution  x**  (•)  £  S{x) 
which  maximizes  ri<:(x(-)),  i.e.,  which  satisfies 

-r^(x)  :=  sup  Tk{x{-))  =  tk(x*(x)) 

x{-)es{x) 

leaves  K\(V[aih{K)  U  C)  through  C.  This  solution  exists  by  Theorem  36  since 
K  is  closed  and  S  is  upper  semicompact.  Next,  we  claim  that  x^  :=  x*^ 
K\Vi8Lb{K),  Otherwise,  if  x^  would  belong  to  the  viability  kernel,  it  could  be 
concatenated  with  a  solution  viable  in  K  for  ever,  so  that  the  initial  state  xq 
would  belong  the  viability  kernel,  which  is  not  the  case. 

Furthermore,  x^*  belongs  to  C,  If  not,  x^  would  belong  to  K\C  which  is 
assumed  to  be  locally  viable.  Then  one  could  associate  with  x*^  £  K\{V\ah{K)  U 
C)  a  solution  y{-)  £  S{x^)  and  T  >  0  such  that  y{r)  £  K\(Viah{K)  U  C)  for  all 
T  £  [0,  Tj.  Concatenating  this  solution  to  x*^(-),  we  obtain  a  solution  viable  in  K 
on  an  interval  [0,t^(x)  +  T],  which  contradicts  the  definition  of  x^(“). 

Therefore  x^  belongs  to  AT  fi  C  so  that  there  exists  x\  £  K  n  R{x^).  □ 
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4  The  Path-Dependent  Viability  Theorem 

Since  the  solution  map  of  a  Marchaud  map  F  X  is  upper  semicompact 

by  Theorem  22,  the  equivalence  between  the  first  and  second  statements  of 
Theorem  34  holds  true. 

The  equivalence  between  the  second  and  the  third  statement  follow  from  the 
following  Haddad’s  Path-Dependent  Viability  Theorem: 

Theorem  41  Assume  that  F  is  Marchaud  and  take  A  >  0.  The  two  following 
statements  hold  true: 

1.  If  Kc  TLxiX)  is  closed,  then  K  is  (globally)  viable  under  F  if  and  only  if 

WcpeK,  F{(p)  nVK{(p)  ^ 

2.  If  C  CK  is  closed,  then  K\C  is  locally  viable  under  F  if  and  only  if 

V(^gK\c,  FMnDKM/0 
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Abstract.  In  this  paper,  we  consider  the  problem  of  stabilizing  the  kine¬ 
matic  model  of  a  car  to  a  general  path  in  the  plane,  subject  to  very  mild 
restrictions.  The  car  model,  although  rather  simplified,  contains  some  of 
the  most  relevant  limitations  that  make  application  of  existing  results 
in  the  literature  impossible:  namely,  the  car  can  only  move  forward,  and 
turn  with  a  bounded  steering  radius;  also,  only  limited  sensory  informa¬ 
tion  is  available. 

The  approach  we  follow  to  stabilization  is  to  adapt  to  the  present  gen¬ 
eral  case  an  optimal  synthesis  approach  successfully  applied  in  our  pre¬ 
vious  work  to  tracking  rectilinear  paths.  Due  to  both  the  nature  of  the 
problem,  and  the  solution  technique  used,  the  analysis  of  the  controlled 
system  involves  a  rather  complex  switching  logic.  Hybrid  formalism  and 
verification  techniques  prove  extremely  useful  in  this  context  to  formally 
proof  stability  of  the  resulting  system,  and  are  described  in  detail  in  the 
paper. 


1  Introduction 

In  this  paper  we  consider  the  design  of  a  control  law  for  path  tracking  by  a 
so-called  Dubins’  model  of  a  car.  Dubins’  cars  are  kinematic  models  of  wheeled 
(nonholonomic)  vehicles  that  move  only  forward  in  a  plane,  and  possess  a  lower- 
bounded  turning  radius.  The  model  is  relevant  to  the  kinematics  of  road  vehicles 
as  well  as  aircraft  cruising  at  constant  altitude,  or  sea  vessels. 

Although  the  design  of  control  techniques  for  nonholonomic  vehicles  has  been 
the  subject  of  extensive  research  recently  (see  e.g.  [10,12,6]),  the  additional  con¬ 
straint  that  the  steering  radius  of  the  vehicle  is  lower  bounded  has  not  been 
explicitly  considered.  However,  such  a  restriction  appears  to  be  crucial  in  mak¬ 
ing  a  kinematic  model  of  a  car  relevant  to  real-world  vehicles  encountered  in 

*  The  work  has  been  conducted  with  partial  support  of  PARADES,  a  Cadence, 
Magneti-Marelli  and  ST-microelectronics  E.E.I.G,  by  CNR  PF-MADESSII  SP3.1.2. 

M.D.  Di  Benedetto,  A.  Sangiovanni-Vincentelli  (Eds.):  HSCC  2001,  LNCS  2034,  pp.  133-146,  2001. 
(c)  Springer- Verlag  Berlin  Heidelberg  2001 
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most  applications.  Another  important  assumption  often  used  in  the  literature  is 
that  the  full  state  of  the  system  is  available  for  measurement,  and  that  the  path 
to  be  tracked  is  entirely  known  in  advance.  Instead,  we  consider  in  this  paper 
the  more  realistic  and  less  demanding  case  that  the  vehicle  can  only  measure 
its  current  distance  and  heading  angle  error  with  respect  to  the  closest  point 
on  the  reference  path  in  the  plane,  where  only  the  sign  of  the  path  curvature  is 
detected. 

The  approach  we  follow  to  stabilization  of  Dubins’  cars  is  to  adapt  to  the 
present  general  case  an  optimal  synthesis  approach  successfully  applied  in  our 
previous  work  to  tracking  rectilinear  paths  [llj.  Due  to  both  the  nature  of  the 
problem,  the  type  of  sensors,  and  the  solution  technique  used,  the  analysis  of  the 
controlled  system  involves  a  rather  complex  switching  logic.  Hybrid  formalism 
(see  [5,14,2])  and  verification  techniques  (see  [8,7,1])  prove  extremely  useful  in 
this  context  to  formally  proof  stability  of  the  resulting  system,  and  are  described 
in  detail  in  the  paper,  which  is  organized  as  follows. 

In  Section  2,  a  hybrid  automaton  that  describes  the  motion  of  the  vehicle 
with  respect  to  the  path  is  introduced,  while  in  Section  3  the  path— tracking  con¬ 
troller  is  developed.  Such  controller,  described  in  detail  in  Section  3.2,  is  obtained 
by  considering  a  local  approximation  of  the  desired  path  with  the  tangent  line, 
and  by  using  a  feedback  controller  designed  for  stabilization  on  straight  paths 
(reported  in  Section  3.1),  The  advantages  of  the  novel  hybrid  path-tracking 
formalization  are  exploited  in  Section  4,  where  the  stability  properties  of  the 
proposed  controller  are  investigated.  By  a  reachability  analysis  in  the  continu¬ 
ous  state  space,  a  finite  state  abstract  representation  of  the  hybrid  closed-loop 
automaton  is  obtained.  Though  this  representation  is  not  a  bisimulation,  but 
rather  a  simulation,  of  the  hybrid  automaton  ([5]),  it  suffices  to  prove  the  sta¬ 
bility  properties  of  the  proposed  control.  It  is  shown  that  the  proposed  hybrid 
feedback  controller  achieves  stabilization  of  the  Dubins’  car  on  a  generic  reference 
path  and  sufficient  conditions  for  global  attr activity  are  derived. 

2  Hybrid  Path  Tracking  Modeling  Using  Switching 
Prenet ’s  Frames 

We  consider  the  kinematic  model  of  a  car  moving  forward  on  a  plane,  which 
was  introduced  by  Dubins  in  [4].  A  configuration  of  the  vehicle  is  defined  by 
an  ordered  pair  {M{x^y)^9)  6  IR^  x  5^,  where  {x^y)  are  the  coordinates  of  a 
reference  point  M  in  the  plane  and  9  is  the  angle  made  by  the  direction  of  the 
car  with  respect  to  the  2:-axis.  The  kinematics  of  the  car  are  described  by 

(X  =  V  cos  9  ^ 

y  =  Vsin9  with  1^1  < -5,  (1) 

where  V  is  the  constant  forward  velocity,  u;  the  is  turning  speed  and  the  input 
constraint  models  a  lower  bound  R  >  0  on  the  turning  radius  of  the  Dubins’  car. 
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The  problem  we  are  concerned  with  is  that  of  steering  the  vehicle  to  a  given 
feasible  path  F,  defined  in  the  arclength  parametrization  by 

r  =  {(x,^)  G  B?  I  (x,y)  =  giP)  for  /?  G  H}  ,  (2) 

with  the  following  conditions: 

A)  y(-)  is  a  class  mapping  from  IR  to  ]R^  and  the  orientation  of  F  is  that 
induced  by  increasing.  /?; 

B)  Let  k(P)  denote  the  extension  by  continuity  from  the  left^of  the  curvature 
of  T,  expressed  as  a  function  of  the  arclength  p.  There  exists  a  positive  real 
Rr  such  that  the  normalized  curvature  k{s)  =  Rk{s)  satisfies 

|k(/3)|  =  R\Km  <  A  ^  c  <  1.  (3) 

C)  Considering  the  open  neighborhood  of  the  path 

Tr  =  {x  e  ]R^  3/9  e  IR,  ||x  -  g(l3)\\  <  iJr}  C  IR^  (4) 

for  all  X  G  7?  there  exists  a  unique  nearest  point  on  F. 

In  order  to  describe  the  motion  of  the  vehicle  with  respect  to  the  reference  path  F 
a  mobile  Frenet’s  frame  associated  to  the  curve  F  is  considered.  Given  a  vehicle 
position  M{x,y)  G  Tr,  the  Prenet’s  frame  St{s)\s^^  is  defined  by  the  tangent, 
the  principal  normal  and  the  binormal  axes  of  the  curve  at  the  point  {x{P),  y{P)) 
of  F,  located  at  the  minimum  distance^  from  M{x,y)  (see  Figure  1).  As  the 
vehicle  moves  with  velocity  the  Frenet’s  frame  St{s)  follows  its  motion  so 

^  By  definition,  k{P)  =  lim^^^-  k{s),  at  points  {x{P),y{P))  where  the  curvature  of  F 
is  not  defined. 

^  Note  that,  by  A),  B)  and  C)  the  Prenet’s  frame  is  well-defined  along  F. 
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downwards  upwards 

k(s)  <  0 


Fig,  2.  Hybrid  automaton  PTHA^  modeling  the  car  in  the  transformed  state  space. 


as  to  keep  it  on  the  principal  normal  axis.  The  arclength  abscissa  s  locates  the 
current  Prenet ’s  frame.  The  tangent  and  the  principal  normal  axes  of  St{s) 
remain  within  the  plane  containing  the  curve,  while  the  binormal  axis  points 
either  upwards,  if  the  local  curvature  of  F  is  counterclockwise  (i.e.  k(s)  >  0), 
or  downwards,  if  the  local  curvature  is  clockwise  (i.e.  k{s)  <  0).  Introduce  the 
transformated  coordinates  {s,y,0),  where: 

-  abscissa  s  defines  the  position  of  the  Prenet’s  frame  along  the  curve; 

-  y  denotes  the  position  of  the  car  along  the  principal  normal  of  St{s)  (lateral 
distance)  normalized  with  respect  to  the  minimum  turning  radius  R; 

-  6  denotes  its  orientation  with  respect  to  the  tangent  axis  of  St{s)  (heading 
angle  error),  with  sign  taken  according  to  the  local  direction  of  the  binormal 
axis  (see  Pigure  1). 

It  can  be  noticed  that  this  coordinate  system  is  similar  to  the  one  used  by 
Samson  [9],  except  for  the  switchings  of  the  Prenet’s  frame.  In  fact,  a  change 
of  curvature  along  the  path  produces  a  jump  of  the  variables  y  and  9  to  the 
symmetric  point  with  respect  to  the  origin  in  the  {y,  ^)-~plane.  The  reason  for 
introducing  such  discontinuity  in  the  model  is  related  to  the  different  behaviors 
that  a  vehicle  with  bounded  curvature  has  when  it  approaches  a  reference  path. 
Indeed,  the  approach  is  apparently  easier  if  the  vehicle  and  the  center  of  cur¬ 
vature  of  the  path  lie  on  the  opposite  sides  of  the  curve^.  This  formulation  will 
turn  out  to  be  useful  in  the  verification  of  the  proposed  path  tracking  controller. 

The  motion  of  the  car  in  the  transformed  state  (s,  y,  9)'^  can  be  described  by 
using  the  formalism  of  hybrid  automata  (see  [5,3]).  The  discrete  nature  of  the 
model  arises  from  the  fact  that  the  Prenet’s  frame  *Sx-(s)  changes  its  orientation 
during  the  motion,  depending  on  the  sign  of  the  curvature  k{s).  The  discrete 
state,  referred  to  as  bin  ,  models  the  two  possible  orientations  of  the  binormal 
axis  of  iST(s(t))  at  time  t  and  assumes  either  the  value  upwards  or  the  value 

^  For  instance,  if  the  vehicle  is  required  to  approach  a  circle  with  curvature  l/R,  then 

it  can  approach  it  only  from  outside. 
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downwards  upwards 

(7r  =  switch 


Fig.  3.  Hybrid  automaton  PTHA2  of  the  vehicle  in  the  reduced  state  space. 


downwards.  Its  initial  value  is:  upwards^  if  «(5(0))  >  0;  downwards,  if  k(5(0))  <  0; 
and  any  of  those,  otherwise.  The  dynamics  the  continuous  states  are  subject 
to  are  obtained  by  geometric  arguments.  The  complete  Path-Tracking  Hybrid 
Automaton,  referred  to  as  PTHA^  ,  is  depicted  in  Figure  2. 

The  specification  for  the  design  of  a  path  tracking  controller  for  the  Dubins’ 
car  can  be  formulated  using  the  hybrid  automaton  PTHA^  ,  which  captures  the 
different  behaviors  of  the  bounded-curvature  vehicle  in  approaching  the  path. 
For  such  hybrid  model,  the  problem  reduces  to  that  one  of  steering  (y,  9)  to 
(0,0). 

Assuming  that  only  the  sign  of  k{s)  is  available  but  not  its  amplitude,  a 
reduced  hybrid  automaton  can  be  considered  for  the  path  tracking  problem. 
The  local  curvature  \k(s)  \  is  replaced  by  an  unknown  input  disturbance  d{t)  the 
path  tracking  controller  has  to  be  robust  to.  By  (3),  disturbance  d{t)  satisfies 

0  <  d{t)  <C<1.  (5) 

The  path  tracking  problem  is  described  in  the  reduced  continuous  state  space 
(y,6).  Curvature  sign  switching  conditions  k{s)  >  0  and  k{s)  <  0  are  modeled 
by  a  discrete  uncontrollable  input  assuming  either  the  value  switch  (when  a 
change  of  curvature  sign  occurs)  or  the  silent  move  e  (otherwise).  The  reduced 
hybrid  automaton,  referred  to  as  PTHA2  ,  is  reported  in  Figure  3. 

In  this  case  the  path  tracking  problem  is  formulated  as  follows: 


Problem  1.  Let  P  as  in  (2)  be  a  feasible  reference  patL  Given  the  hybrid  au¬ 
tomaton  PTHA2  ,  find  a  feedback  control  law  uj{hin,  {y,0))  satisfying  curvature 
constraint  (1)  such  that,  from  any  initial  state  (6«no,  (yo,  ^0))  the  trajectory 
{y{t),9{t))  converges  to  the  origin  under  the  action  of  any  unknown  disturbance 
d{t),  bounded  as  in  (5),  and  any  sequence  of  uncontrollable  events  (Jr* 
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Table  1.  Partition  of  domain  used  to  define  the  shortest  path  synthesis. 


3  Hybrid  Path-Tracking  Feedback  Controller 

3.1  Optimal  Feedback  Control  for  Line  Tracking 

In  [11],  the  problem  of  driving  the  Dubins’  car  to  a  straight  path  has  been 
considered.  An  optimal  feedback  control  that  minimizes  the  length  travelled  by 
the  vehicle  to  reach  the  specified  path  was  deviced.  Define  ajv(y,  =  y  4-  1  + 
cos(0)  and  ap{y,9)  =  y  —  I  ~  cos{0).  The  optimal  feedback  control  presented 
in  [11]  is  defined  inside  the  region 


(7N{yJ)  <  0  A  [tt,  Itt)  V 
<  0  A  (f,7r)  V 
V 

CTNiy,  ^)  >  0  A  9  G  [-7T,  -|)  V 
>  0  A  ^  G  (“Itt,  -tt) 


(6) 


in  the  state  space  (y,  ^),  which,  modulo  27r  angles  on  0,  corresponds  to  the  whole 
space  (see  Figure  5).  The  optimal  controller  is  described  by  three  modes. 
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•  go  straight,  where  a;  =  0 

•  turn.right,  where  to  =  (7) 

•  turnJeft,  where  oj  =  , 

which  are  chosen  as  follows 

[gostraight,  if  {y,  0)  C  i?^]  A  [turn.right,  if  (y,  9)  G  Q~\  A  [tumJeft,  if  {y,  9)  G  O'^] 

(8) 

where  the  partition  i?°  U  U  of  domain  is  defined  as  in  Table  1, 

In  Figure  5  the  boundaries  between  the  subsets  of  the  partition  U  i?“  U  i7+ 
are  represented  by  dotted  lines,  and  the  direction  of  motion,  when  the  reference 
path  is  a  straight  line  i.e.  d  =  0,  is  represented  by  directed  curves. 

3.2  Feedback  Tracking  Control  for  Generic  Path 

In  this  section  a  hybrid  feedback  controller  that  solves  Problem  1  is  derived 
from  the  one  reported  in  the  previous  section.  The  hybrid  model  of  the  vehicle 
PTHA2  is  characterized  by  the  two  modes:  upwards  and  downwards.  In  mode 
downwards  input  cj  appears  with  opposite  sign  with  respect  to  mode  upwards. 
Since  the  controller  modes  in  (8)  has  been  set  assuming  an  upwards  binormal 
axis  then,  the  controller  modes  turn-right  and  tum_left  have  to  be  switched  when 
the  vehicle  is  in  mode  downwards.  Hence,  for  a  generic  feasible  path  T,  the  full- 
state  feedback  controller  is  defined  in  {upwards,  downwards}  x  by  setting 

the  controller  modes  as  follows 

•  go-straight,  if  {bin,  {y,9))  G  {upwards,  downwards}  x  17® 

•  turn-right,  if  {bin,  {y,9))  G  {upwards  x  j7“)  V  {bin,  {y,9))  G  {downwards  x  17+) 

•  turn-left,  if  {bin,  {y,9))  G  {upwards  x  17+)  V  (6m,  {y^9))  G  {downwards  x  17“) 

(9) 

where  17®,  17~  and  17+  are  as  in  Table  1.  The  closed-loop  hybrid  automaton 
CLHA  obtained  by  applying  the  feedback  (7), (9)  to  the  vehicle  hybrid  automaton 
PTHA2  is  depicted  in  Figure  4.  According  to  (9)  and  (8),  CLHA  has  a  discrete 
state  mode  that  assumes  values  in  the  set  O  =  {zero,  negative,  positive},  as  follows 

•  mode  =  zero  if  {y,  9)  G  17® 

•  mode  —  negative  if  {y,  9)  G  Q~  (10) 

•  mode  =  positive  if  {y,  9)  G  17+ . 

The  initial  state  (modeo,  (yo?  ^0))  of  the  hybrid  automaton  CLHA  has  to  sat¬ 
isfy  (10). 

The  coordinate  transformation  {x,y,9)  {s,y,9)  becomes  singular  when 

the  vehicle  lies  on  the  center  of  the  local  osculating  circle  to  the  path  P.  That  is 

if,  at  some  time  t,  y{L)  |«(s(^)|  =  1,  or  equivalently  y{t)  d{t)  =  1.  For  any  initial 

configuration  {M{xo,yo),9o),  with  M{xo,yo)  G  Tr  as  in  (4),  the  corresponding 
state  {yo,9o)  satisfies  yo  <  C~^.  Further,  since  by  (5)  d  <  C,  then  ^od  <  1  at 
the  given  initial  condition.  However,  to  ensure  that 

yd  <  1  i.e.  1  —  >  0 


(11) 
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Fig.  4.  Hybrid  model  of  the  closed-loop  system  CLHA  . 

will  hold  along  all  the  trajectories  of  CLHA  ,  we  need  to  further  restrict  the 
admissible  initial  vehicle  configurations,  in  terms  of  its  initial  orientation  ^o- 

Proposition  1.  Let  the  continuous  disturbance  d  be  bounded  to  belong  to  the 
interval  [0,  C],  with 

C  <  0.5  .  (12) 

Then,  (11)  is  satisfied  along  all  trajectories  of  CLHA  provided  that  the  initial 
configuration  {modeo,{yo,9o))  is  such  that 

(yoJo)  e  I  |if|  <  C"!  -  1  +  |cos(0)|| .  (13) 

The  proof  of  the  above  proposition  is  not  reported  due  to  space  limitation. 

Note  that,  for  initial  configurations  satisfying  (13)  we  have  Af(xo,  i/o)  €  7r  as 
in  (4).  By  Proposition  1,  if  a  reference  path  P  has  minimum  radius  of  curvature 
Rr  greater  than  twice  the  minimum  turning  radius  R  of  the  vehicle,  then  for  any 
initial  configuration  {M{xo,yo),9Q),  with  lateral  position  and  orientation  errors 
bounded  to  belong  to  as  in  (13),  condition  (11)  is  ensured. 
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4  Verification  of  the  Hybrid  Path-Tracking  Controller 


In  this  section  the  behavior  of  the  hybrid  automaton  CLHA  is  analyzed  by  in¬ 
troducing  an  equivalence  relation  in  the  hybrid  state  space  O  x  and  by 

computing  the  corresponding  quotient  system  (see  [5]). 

Consider  the  partition  of  the  domain  (6)  given  by  the 

24  subsets  •  •  • ,  ,  lsl^^\  o|,  defined  in  Table  1,  with  rl^^^  and 

1^^^  replaced  by  We  say  that  (modei^(ifi,  ^1)),  (mode2,  (^2,  ^2))  are 

equivalent,  i.e.  (modei,  (^1, ^1))  ~  {fnode2,(y2i^2))^  iff  C  p,  for  some 

p  G  implies  (^2,^2)  C  p.  We  associate  to  the  corresponding  quotient 

space  x  •••,0x0}  a  nonderministic  finite  state  machine,  re¬ 

ferred  to  as  FSM  pj'Q  ,  with  states  corresponding  to  the  equivalence  classes  in 
Q'"  (labeled,  with  a  slight  abuse  of  notation,  •  *  • ,  O).  The  next-state  func¬ 
tion  of  FSM prpQ  is  defined  as  follows:  for  any  Qi,Q2  C  Q"",  a  transition  from 
Qi  to  Q 2  occurs  iff  there  exists  an  arc  of  trajectory  of  the  hybrid  automoton 
CLHA  from  some  {modei,  (^1,  ^1))  C  Qi  to  some  {mode2,  (^2?  ^2))  ^  Q2,  for  some 
discrete  disturbance  ar  and  some  continuous  disturbance  d. 

Proposition  2.  Given  the  hybrid  system  CLHA  ,  if  the  discrete  disturbance  ar 
takes  always  the  value  e,  then,  for  any  initial  hybrid  state  {mode,(yo,0o))  G 
O  X  X^~  as  in  (13),  under  the  action  of  any  disturbance  d  bounded  as  in  (5) 
with  C  as  in  (12),  we  have: 

—  the  quotient  system  obtained  from  the  equivalence  relation  ~  is  the  finite 
state  machine  FSMppQ  depicted  in  Figure  5; 

—  an  upper  bound  for  the  space  travelled  by  the  origin  of  the  Frenet’s  frame 
along  the  path  F,  when  the  hybrid  state  is  in  a  given  equivalence  class  is 
represented  by  the  weight  associated  to  exiting  arc; 

—  the  quotient  system  FSMppQ  remains  in  each  equivalence  class  a  bounded 
amount  of  time,  except  for  the  equivalent  class  O  where  (y^O)  =  (0,0). 

The  proof  of  the  above  proposition,  which  is  based  on  reachability  analysis,  is 
not  reported  due  to  space  limitation. 

If  the  reference  path  F  has  curvature  always  of  the  same  sign,  the  convergence 
of  the  Dubins’  car  to  the  path  is  guaranteed  by: 

Corollary  1.  If  the  reference  path  F  has  curvature  always  of  the  same  sign  and 
amplitude  lower  than  the  hybrid  feedback  control  (7)  and  (9),  ensures  the 
tracking  of  F  for  any  initial  vehicle  configuration  in  the  domain 
The  origin  of  the  Frenet’s  frame  covers  at  most  a  distance  of 


1  -f  f  TT  -h  g  if  C  G  [0,  g^) 
4-l-77r-f-^  ifCGig:^,!) 


(14) 


along  the  reference  path  F  before  the  vehicle  approaches  it  with  correct  orienta¬ 
tion. 
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Fig.  5.  On  the  left:  shortest  paths  synthesis  when  d  =  0.  On  the  right:  quotient  system 
FSM prpQ  representing  the  behavior  of  the  closed-loop  hybrid  system  CLHA  ,  when 

<Jr  =  e. 


The  proof  of  the  above  corollary  is  obtained  by  computing  the  longest  path  to 
the  node  O. 

By  Proposition  2,  if  T  is  a  straight  line  then  the  closed— loop  system  enforces 
sliding  motions  (see  [13]  for  a  tutorial)  in  the  space  {y,0)  on  the  lines  sr,  si 
and  the  arcs  1^^^  1^^^  until  the  origin  is  reached.  If  the  reference  path 

r  is  not  a  straight  line,  sliding  motions  are  enforced  only  on  the  lines  sr,  si, 
on  the  arcs  and  on  a  piece  of  the  arc  Under  ideal  sliding  motion, 

around  the  origin  the  control  uj  switches  at  infinite  frequency  between  ^ ,  0  and 
—  The  mean  value  of  such  control  (i.e.  the  equivalent  control)  is  the  signal 
kV  that  makes  the  car  follows  the  reference  path  V  with  velocity  V.  In  the  real 
implementation  smoothing  techniques  are  applied  to  avoid  the  chattering  of  the 
control  signal  between  the  three  values  ^,0  and  — 

The  behavior  of  the  closed-loop  system  CLHA  under  the  action  of  the  dis¬ 
crete  disturbance  (7^  is  characterized  by  the  following  propositions. 

Proposition  3.  Given  an  initial  condition  (yo,^o)  i'fi  the  open  neighborhood  of 
the  origin 

:  lyl  <  1,-arccos  Q  “  f )  <  ^  <  arccos  Q  +  |  (15) 
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Fig.  6.  On  the  left:quotient  system  representing  the  behavior  of  the  closed- 

loop  hybrid  system  CLHA  ,  when  the  intial  state  belongs  to  O  x  On  the  right: 

regions  in  the  domain  where  W  >  0. 


(see  Figure  6),  the  hybrid  closed-loop  system  CLHA  keeps  the  continuous-time 
trajectory  {y(t),9{t))  inside  under  any  disturbance  d{t)  bounded  as  in  (5) 

and  any  sequence  of  events 

Due  to  space  limitation,  the  proof  of  the  above  proposition  is  not  reported. 

Proposition  4.  If  the  reference  path  F  is  such  that  changes  in  the  curvature 
sign  are  at  distance  greater  than  (5  +  f  )i?  along  if  then  the  hybrid  feedback 
control  (7),  with  modes  chosen  according  to  (9)  stabilizes  the  Dubins’  car  along 
the  reference  path  F. 

Proof  The  set  defined  in  (15)  is  such  that 

A/'(-  C  O  U  U  U  U  U  U  . 

Since,  by  Proposition  3,  is  a  robust  invariant  set  for  the  closed-loop  hy¬ 
brid  system  CLHA  ,  then,  if  we  restrict  our  attention  to  the  domain  the 

transitions  from  Isr^^^and  from  rsl^^^in  the  quotient  system 

FSM pj'Q  should  be  removed.  Furthermore,  notice  that,  under  the  action  of  the 
discrete  disturbance  ar  —  switch,  the  reset  y  :=  —y  and  9  :=  —6  introduces  the 
mutual  transitions  and  Hence,  in  the 

presence  of  the  discrete  disturbance  ar  and  for  any  disturbance  d  as  in  (5),  when 
the  initial  state  belongs  to  O  x  the  quotient  system  FSMppQ  obtained 

fi’om  the  equivalent  relation  ~  is  as  in  Figure  6. 

To  analyse  the  convergence  of  the  trajectories  to  O,  introduce  the  function 

W{y,9)  =  \(f  +  e^). 


(16) 
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W{y,e)  has  the  property  that  if,  at  time  t  =  t,ar  =  switch  then  W{y{t),  e{t))  = 
W{y{t~),9(t~)).  The  derivative  with  respect  to  time  of  W  evaluates  to 

W{yJ)=  ysm{§)-0^^-ew  (17) 

where  w  =  0,-1,  and  1  in  mode  zero,  negative^  and  positive^  respectively.  The 
study  of  the  sign  of  W{y,B)  is  extended  to  the  entire  domain  '^(^y§y  Under 
assumption  (11),  multiplying  (17)  by  ^(1  -  if d),  we  have 

>  0  ^  p{y,  9)  =  d  sm{9)  —  9cos{9)  +  sin(0)  -  w9  >  0, 

for  some  disturbance  d  bounded  as  in  (5).  Hence,  for  any  {y,  9)  such  that 

m(y,<9)  =  ysm{9)  -w9>0, 

there  exists  d  as  in  (5)  such  that  p{y,  ^)  >  0  and  >  0.  Otherwise,  if  (y,  9)  is 
such  that  7/1  (y,  9)  <  0,  then  there  exists  d  as  in  (5)  such  that  VP  >  0  if  and  only 
if  /x(y,  9)  is  positive  for  d  =  1.  That  is,  if 

miy,  0)  =  -  sm{9)y^  +  sin(^)  +  ru9j  y  -  ^cos(^)  ~{-w9  >0. 

The  regions  in  the  domain  0)  where  function  (16)  locally  increases  are  re¬ 
ported  in  Figure  6.  Such  regions  are  delimited  by  the  curves  7/1  (y,^)  =  0  and 
??2(^}^)  =  0-  By  (17),  the  continuous  disturbance  d  that  maximizes  W{t)  is 

^  f  1  if  9cos{9)  <  0  i.e.  9  e  (-f  ,0)  U  (f ,  |7r) 

\0  if0cos(0)>O  i.e.  ^  €  (“Itt, -|)  U  (0,  |)  ■  ^  ^ 

Consider  an  initial  condition  (yo,  ^0)  in  a  neighborhood  of  the  origin  contained  in 
At  the  initial  time,  the  hybrid  model  CLHA  is  in  mode  negative. 
Let  us  assume  that  ar  ~  for  the  moment,  and  let  us  analyse  the  evolution 
of  the  hybrid  model  CLHA  (see  Figure  6).  Under  the  action  of  the  worst  dis¬ 
turbance  (18),  the  trajectory  {y{t),9(t))  originating  from  (yo,^o)  reaches  the 
curves  rU).  First  W{t)  decreases  (in  rl^^'^^),  then  it  increases  (in  Hence, 

mode  switches  to  positive.  W{t)  decreases  (in  the  first  part  of  lr^^‘^^),  and  it  in¬ 
creases  again  later  on  (in  and  until  (y(t),  9{t))  reaches  Finally, 

following  a  sliding  motion  along  the  curve  0{t))  reaches  the  origin. 

Along  this  trajectory  W (t)  assume  two  local  maxima,  which  correspond  to 
the  intersections  of  1^^^  and  and  two  local  minima:  the  first  on  the  line  ^  =  0 
when  y  >  0,  and  the  second  inside  region  Let  S  ~  ||(yo,^o)||-  Since  the 

trajectory  {y{t),9{t))  is  continuous  with  respect  to  the  initial  condition  (yo^Bo), 
then  there  exist  two  continuous  functions  Cmax,  Cmin  :  H  H  such  that 

maxmp||(y(t),i9(0)||  =  Cmax(<^),  mjnmm  ||(y(t),  (9(i))||  =  Cmin(^).  (19) 
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Further,  since  the  local  maximum  and  minimum  points  tend  to  the  origin  as 
11(^05 ^o)||  tends  to  zero,  then  lim^-^o Cmax(^)  =  0  and  lim5_).o  Cmin(<^)  =  0- 

Suppose  now  that  a  discrete  disturbance  (Jr  =  switch  occurs  at  the  precise 
time  i  at  which  (y{t~),9{i~))  is  opposite  to  {yo,9q)  with  respect  to  the  ori¬ 
gin.  Then,  the  state  (y(i~),9{t~))  is  reset  to  {y{t),9{t})  =  {-yii~), -9(t~))  G 
which  lies  on  the  same  line  to  the  origin  of  (yo,^o)-  If  I^(yo,^o)  > 

W{y{l),9{^)  =  W{-y{t~),—9{i~))  then  the  convergence  is  preserved.  But,  if 

then,  under  the  action  of  the 
discrete  disturbance  ar  =  switch^  the  state  {y,  9)  is  reset  to  a  point  farther  away 
from  the  origin  than  the  initial  state  (^Oj^o)  and  convergence  can  be  lost. 

However,  if  the  reference  path  F  is  such  that  changes  in  the  curvature  sign 
are  at  a  distance  greater  than  {b  +  ~)R  along  it,  between  to  successive  actions  of 
the  discrete  disturbance  cr^,  the  state  (y,  9)  has  enough  time  to  reach  the  origin. 
In  fact,  assuming  that,  in  the  worst  case,  {y{t),9{F))  G  an  upper 

bound  on  the  length  the  arc  of  F  spanned  by  the  origin  of  the  Prenet ’s  frame  as 
{y{t)^9{t))  converges  to  the  origin,  is  given  by  L(l^^^)+ 

^(lr(i-2))_|-  that,  according  to  the  weights  reported  on  the 

quotient  system  FSM pj'Q  depicted  in  Figure  5,  evaluates  to  (5  -f  ^)i2. 

To  prove  the  robust  stabilization  of  the  car  along  the  reference  path  F  we  have 
to  show  that  for  any  e  >  0,  there  exists  5  >  0  such  that  any  trajectory  {y_{t),  9{t)) 
of  the  hybrid  system  CLHA  ,  originating  from  any  (yo,^o)  with  ||(yo,^o)||  <Ji 
we  have  ||(y(t),  ^(0)11  <  Given  any  €  >  0,  consider  any  initial  condition  (yo,  ^o) 
with 

||{yo, eo)||  <  <5  =  Cix(QUClx(^)))-  (20) 

The  trajectory  {y{t),9{t))  evolves  inside  a  ball  of  radius  Cmin(CmL(^))*  H  ^ 
disturbance  <7^  =  switch  occurs  at  some  time  t,  then  the  state  is  reset  to 
{y{t),9{F))  =  {—y{i~),-9{i~))  G  Af^y^ey  In  evolution  for  t  t  the  tra¬ 
jectory  reaches  the  origin  before  a  further  discrete  disturbance  will  show  up. 
Morever,  since  ||(ji(i), e(t)||  <  C“iUCmL(«))  then,  the  trajectory  {y(t),e{t))  for 
t  >  t  does  not  exit  a  ball  of  radius  Cmax(Cmlx(^))  —  Then,  the  hybrid  feedback 
control  (7),  with  modes  chosen  according  to  (9)  robustly  stabilizes  the  car  along 
the  reference  path  F. 


5  Conclusions 

In  this  paper,  we  have  used  modern  techniques  developed  for  hybrid  systems 
simulation  and  verification  to  solve  and  prove  stability  of  a  control  technique 
for  an  interesting  problem,  that  is  route  tracking  by  nonholonomic  vehicles  with 
bounds  on  the  curvature  and  limited  sensory  information.  The  proposed  con¬ 
troller  is  reminiscent  of  a  synthesis  proposed  elsewhere  for  an  optimal  control 
problem  to  track  straight  routes,  whose  generalization  to  generic  routes  turned 
out  to  be  difficult  to  analyze  otherwise.  We  believe  that  this  case  study,  besides 
its  intrinsic  interest  in  applications,  also  has  a  value  in  showing  the  potential  of 
hybrid  systems  analysis  techniques  as  applied  to  complex  control  problems. 
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Abstract.  This  paper  introduces  the  model  of  linearly  priced  timed 
automata  as  an  extension  of  timed  automata,  with  prices  on  both 
transitions  and  locations.  For  this  model  we  consider  the  minimum-cost 
reachability  problem:  i.e.  given  a  linearly  priced  timed  automaton  and  a 
target  state,  determine  the  minimum  cost  of  executions  from  the  initial 
state  to  the  target  state.  This  problem  generalizes  the  minimum-time 
reachability  problem  for  ordinary  timed  automata.  We  prove  decid¬ 
ability  of  this  problem  by  offering  an  algorithmic  solution,  which  is 
based  on  a  combination  of  branch-and-bound  techniques  and  a  new 
notion  of  priced  regions.  The  latter  allows  symbolic  representation  and 
manipulation  of  reachable  states  together  with  the  cost  of  reaching  them. 

Keywords:  Timed  Automata,  Verification,  Data  Structures,  Algo¬ 
rithms,  Optimization. 


1  Introduction 

Recently,  real-time  verification  tools  such  as  UPPAAL  [14],  Kronos  [7]  and 
HyTech  [11],  have  been  applied  to  synthesize  feasible  solutions  to  static  job- 
shop  scheduling  problems  [9,13,18].  The  basic  common  idea  of  these  works  is  to 
reformulate  the  static  scheduling  problem  as  a  reachability  problem  that  can  be 
solved  by  the  verification  tools.  In  this  approach,  the  timed  automata  [3]  based 
modeling  languages  of  the  verification  tools  serve  as  the  basic  input  language 
in  which  the  scheduling  problem  is  described.  These  modeling  languages  have 
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proven  particularly  well-suited  in  this  respect,  as  they  allow  for  easy  and  flexible 
modeling  of  systems,  consisting  of  several  parallel  components  that  interact  in  a 
time-critical  manner  and  constrain  each  other’s  behavior  in  a  multitude  of  ways. 

In  this  paper  we  introduce  the  model  of  linearly  priced  timed  automata  and 
offer  an  algorithmic  solution  to  the  problem  of  determining  the  minimum  cost 
of  reaching  a  designated  set  of  target  states.  This  result  generalizes  previous 
results  on  computation  of  minimum-time  reachability  and  accumulated  delays 
in  timed  automata,  and  should  be  viewed  as  laying  a  theoretical  foundation  for 
algorithmic  treatments  of  more  general  optimization  problems  as  encountered  in 
static  scheduling  problems. 

As  an  example  consider  the  very  simple  static  scheduling  problem  repre¬ 
sented  by  the  timed  automaton  in  Fig.  1  from  [17],  which  contains  5  ’tasks’ 
{AyB,C,D^E}.  All  tasks  are  to  be  performed  precisely  once,  except  task  C, 
which  should  be  performed  at  least  once.  The  order  of  the  tasks  is  given  by  the 
timed  automaton,  e.g.  task  B  must  not  commence  before  task  A  has  finished.  In 
addition,  the  timed  automaton  specifies  three  timing  requirements  to  be  satis¬ 
fied:  the  delay  between  the  start  of  the  first  execution  of  task  C  and  the  start  of 
the  execution  of  E  should  be  at  least  3  time  units;  the  delay  between  the  start  of 
the  last  execution  of  C  and  the  start  of  D  should  be  no  more  than  1  time  unit; 
and,  the  delay  between  the  start  of  B  and  the  start  of  D  should  be  at  least  2 
time  units,  each  of  these  requirements  are  represented  by  a  clock  in  the  model. 
Using  a  standard  timed  model  checker  we  are  able  to  verify  that  location  E  of 


Fig.  1.  Timed  automata  model  of  scheduling  example. 


the  timed  automaton  is  reachable.  This  can  be  demonstrated  by  a  trace  leading 
to  the  location^: 

(A,  0, 0, 0)  (B,  1, 1. 1)  (C,  2, 1, 1)  (0. 4, 3, 3)  A  (£,  4, 3, 3)  (1) 

The  above  trace  may  be  viewed  as  a  feasible  solution  to  the  original  static 
scheduling  problem.  However,  given  an  optimization  problem,  one  is  often  not 
satisfied  with  an  arbitrary  feasible  solution  but  insist  on  solutions  which  are  opti¬ 
mal  in  some  sense.  When  modeling  a  problem  like  this  one  using  timed  automata 
an  obvious  notion  of  optimality  is  that  of  minimum  accumulated  time.  For  the 

^  Here  a  quadruple  {X,Vx,Vy^Vz)  denotes  the  state  of  the  automaton  in  which  the 
control  location  is  X  and  where  Vx,Vy  and  Vz  give  the  values  of  the  three  clocks 
X,  y  and  2.  The  transitions  labelled  r  are  actual  transitions  in  the  model,  and  the 
transitions  labelled  e{d)  represents  a  delay  of  d  time  units. 
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Fig.  2.  A  linearly  priced  timed  automaton. 


timed  automaton  of  Fig.  1  the  trace  of  (1)  has  an  accumulated  time-duration  of 
4.  This,  however,  is  not  optimal  as  witnessed  by  the  following  alternative  trace, 
which  by  exploiting  the  looping  transition  on  C  reaches  E  within  a  total  of  3 
time-units^: 

(A,  0, 0, 0)  -4 (C,  2, 2, 2)  4-  (C,  2, 0. 2)  4-^  {D,  3, 1, 3)  4  {E, 3, 1, 3)  (2) 

In  [4]  algorithmic  solutions  to  the  minimum-time  reachability  problem  and  the 
more  general  problem  of  controller  synthesis  has  been  given  using  a  backward 
fix-point  computation.  In  [17]  an  alternative  solution  based  on  forward  reacha¬ 
bility  analysis  is  given,  and  in  [5]  an  algorithmic  solution  is  offered,  which  applies 
branch-and-bound  techniques  to  prune  parts  of  the  symbolic  state-space  which 
are  guaranteed  not  to  contain  optimal  solutions.  In  particular,  by  introducing 
an  additional  clock  for  accumulating  time-elapses,  the  minimum-time  reachabil¬ 
ity  problem  may  be  dealt  with  using  the  existing  efficient  data  structures  (e.g. 
DBMs  [8],  CDDs  [15]  and  DDDs  [16])  already  used  in  the  real-time  verification 
tools  Uppaal  and  Kronos  for  reachability.  The  results  of  the  present  paper 
also  extends  the  work  in  [2]  which  provides  an  algorithm  for  computing  the 
accumulated  delay  in  a  timed  automata. 

In  this  paper,  we  provide  the  basis  for  dealing  with  more  general  optimiza¬ 
tion  problems.  In  particular,  we  introduce  the  model  of  linearly  priced  timed 
automata^  as  an  extension  of  timed  automata  with  prices  on  both  transitions 
and  locations:  the  price  of  a  transition  gives  the  cost  for  taking  it  and  the  price 
on  a  location  specifies  the  cost  per  time-unit  for  staying  in  that  location.  This 
model  can  capture  not  only  the  passage  of  time,  but  also  the  way  that  e.g.  tasks 
with  different  prices  for  use  per  time  unit,  contributes  to  the  total  cost.  Figure  2 
gives  a  linearly  priced  extension  of  the  timed  automaton  from  Fig.  1.  Here,  the 
price  of  location  D  is  set  to  /3  and  the  price  on  all  other  locations  is  set  to  1  (thus 
simply  accumulating  time).  The  price  of  the  looping  transition  on  C  is  set  to  a, 
whereas  all  other  transitions  are  free  of  cost  (price  0).  Now  for  (a,^)  =  (1,3) 
the  costs  of  the  traces  (1)  and  (2)  are  8  and  6,  respectively  (thus  it  is  cheaper 
to  actually  exploit  the  looping  transition).  For  (a,/3)  =  (2,2)  the  costs  of  the 
two  traces  are  both  6,  thus  in  this  case  it  is  immaterial  whether  the  looping 
transition  is  taken  or  not.  In  fact,  the  optimal  cost  of  reaching  E  will  in  general 

^  In  fact,  3  is  the  minimum  time  for  reaching  E. 
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be  the  minimum  of  2  +  2  *  ^  and  3  +  a,  and  the  optimal  trace  will  include  the 
looping  transition  on  C  depending  on  the  particular  values  of  a  and  /?, 

In  this  paper  we  deal  with  the  problem  of  determining  the  minimum  cost 
of  reaching  a  given  location  for  linearly  priced  timed  automata.  In  particular, 
we  offer  an  algorithmic  solution  to  this  problem^.  In  contrast  to  minimum-time 
reachability  for  timed  automata,  the  minimum-cost  reachability  problem  for  lin¬ 
early  priced  timed  automata  requires  the  development  of  new  data  structures 
for  symbolic  representation  and  the  manipulation  of  reachable  sets  of  states  to¬ 
gether  with  the  cost  of  reaching  them.  In  this  paper  we  put  forward  one  such 
data  structure,  namely  a  priced  extension  of  the  fundamental  notion  of  clock 
regions  for  timed  automata  [3] . 

The  remainder  of  the  paper  is  structured  as  follows:  Section  2  formally  intro¬ 
duces  the  model  of  linearly  priced  timed  automata  together  with  its  semantics. 
Section  3  develops  the  notion  of  priced  clock  regions,  together  with  a  number  of 
useful  operations  on  these.  The  priced  clock  regions  are  then  used  in  Section  4  to 
give  a  symbolic  semantics  capturing  (sufficiently)  precisely  the  cost  of  executions 
and  used  as  a  basis  for  an  algorithm  solution  to  the  minimum-cost  problem.  Fi¬ 
nally,  in  Section  5  we  give  some  concluding  remarks.  We  refer  the  read  to  [6]  for 
the  proofs  not  included  in  this  paper. 


2  Linearly  Priced  Timed  Automata 

In  this  section,  we  introduce  the  model  of  linearly  priced  timed  automata,  which 
is  an  extension  of  timed  automata  [3]  with  prices  on  both  locations  and  transi¬ 
tions.  Dually,  linearly  priced  timed  automata  may  be  seen  as  a  special  type  of 
linear  hybrid  automata  [10]  or  multirectangular  automata  [10]  in  which  the  ac¬ 
cumulation  of  prices  (i.e.  the  cost)  is  represented  by  a  single  continuous  variable. 
However,  in  contrast  to  known  undecidability  results  for  these  classes,  minimum- 
cost  reachability  is  computable  for  linearly  priced  timed  automata^. 

Let  <7  be  a  finite  set  of  clocks.  Then  IS(C)  is  the  set  of  formulas  obtained 
as  conjunctions  of  atomic  constraints  of  the  form  x  ixi  n  where  x  G  C,  n  is 
natural  number,  and  M  G  {<,<,—,>,>}.  Elements  of  8(C)  are  called  clock 
constraints  over  C.  Note  that  for  each  timed  automaton  that  has  constraints  of 
the  form  x  —  y  M  c,  there  exists  a  strongly  bisimilar  timed  automaton  with  only 
constraints  of  the  form  x  ixi  c.  Therefore,  the  results  in  this  paper  are  applicable 
to  automata  having  constraints  of  the  type  x  —  y  IX3  c  as  well. 

Definition  1  (Linearly  Priced  Timed  Automaton).  A  Linearly  Priced 
Timed  Automaton  (LPTA)  over  clocks  C  and  actions  Act  is  a  tuple 
(LjIq^  I,  P)  where  L  is  a  Unite  set  of  locations,  Iq  is  the  initial  location, 
E  C  L  X  B(C)  X  Act  X  V(C)  x  L  is  the  set  of  edges,  I  :  L  ^(C)  £^ssigns 

^  Thus  settling  an  open  problem  given  in  [4]. 

An  intuitive  explanation  for  this  is  that  the  additional  (cost)  variable  does  not  in¬ 
fluence  the  behavior  of  the  automata. 
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X  >  1 


Fig.  3.  An  example  LPTA.  ' 


invariants  to  locations,  and  P  :  (L  U  E)  N  assigns  prices  to  both  locations 
and  edges.  In  the  case  of  {I,  g,  a,  r,  /')  G  E,  we  write  I  I'. 

Formally,  clock  values  are  represented  as  functions  called  clock  assignments 
from  C  to  the  non-negative  reals  M>o-  We  denote  by  the  set  of  clock  assign¬ 
ments  for  C  ranged  over  by  u,  u'  etc.  We  define  the  operation  u'  ~  [r  0]u 
to  be  the  assignment  such  that  u'{x)  =  0  if  x  E  r  and  u{x)  otherwise,  and  the 
operation  u'  =  u  d  to  he  the  assignment  such  that  u'{x)  =  u{x)  4-  d.  Also,  a 
clock  valuation  u  satisfies  a  clock  constraint  g,u  E  g,  if  u{x)  >3  n  for  any  atomic 
constraint  x  n  in  g.  Notice  that  the  set  of  clock  valuations  satisfying  a  guard 
is  always  a  convex  set. 

The  semantics  of  a  LPTA  A  is  defined  as  a  transition  system  with  the  state- 
space  L  X  R^,  with  initial  state  {Iq,uo)  (where  uq  assigns  zero  to  all  clocks  in 
C),  and  with  the  following  transition  relation: 

-  {l,u)  {l,u  +  d)ifu  +  d€  I{1),  and  p  =  P{1)  *  d. 

—  (1,  u)  (/',  u')  if  there  exists  o,  r  such  that  I  u  E  g,  u'  =  \r  Olu, 

u'  El{l')  and  p  =  P{{l,g,  a,  r,  I')). 

Note  that  the  transitions  are  decorated  with  two  labels:  a  delay-quantity  or  an 
action,  together  with  the  cost  of  the  particular  transition.  For  determining  the 
cost,  the  price  of  a  location  gives  the  cost  rate  of  staying  in  that  location  (per 
time  unit),  and  the  price  of  a  transition  gives  the  cost  of  taking  that  transition. 
In  the  remainder,  states  and  executions  of  the  transition  system  for  LPTA  A 
will  be  referred  to  as  states  and  executions  of  A. 

Definition  2  (Cost).  Let  a  =  {lo,uo)  (/„,w„)  be  a 

finite  execution  of  LPTA  A.  The  cost  of  a,  cost(a),  is  the  sum  ...  ^n}Pi- 

For  a  given  state  the  minimal  cost  of  reaching  mincost((/,  u)), 

is  the  infimum  of  the  costs  of  finite  executions  ending  in  (l,u).  Similarly,  the 

minimal  cost  of  reaching  a  location  I,  mincost(/),  is  the  infimum  of  the  costs  of 
finite  executions  ending  in  a  state  of  the  form  {l,u). 


Example  1.  Consider  the  LPTA  of  Fig.  3.  The  LPTA  has  a  single  clock  x,  and 
the  locations  and  transitions  are  decorated  with  prices.  A  sample  execution  of 
this  LPTA  is  for  instance: 


(AO) 


r,5 


(41.5)  1.5) -i4(C,  1.5) 


6(1.5), 4.5 
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The  cost  of  this  execution  is  10.5.  In  fact,  there  are  executions  with  cost  arbitrar¬ 
ily  close  to  the  value  7,  obtainable  by  avoiding  delay  in  location  A,  and  delaying 
just  long  enough  in  location  B.  Due  to  the  infimum  definition  of  mincost,  it  fol¬ 
lows  that  m incost (C)  =  7.  However,  note  that  because  of  the  strict  comparison 
in  the  guard  of  the  second  transition,  no  execution  actually  achieves  this  cost.  □ 


3  Priced  Clock  Regions 

For  ordinary  timed  automata,  the  key  to  decidability  results  has  been  the  valu¬ 
able  notion  of  region  [3].  In  particular,  regions  provide  a  finite  partitioning  of 
the  uncountable  set  of  clock  valuations,  which  is  also  stable  with  respect  to  the 
various  operations  needed  for  exploration  of  the  behavior  of  timed  automata  (in 
particular  the  operations  of  delay  and  reset). 

In  the  setting  of  linearly  priced  timed  automata,  we  put  forward  a  new  ex¬ 
tended  notion  of  priced  region.  Besides  providing  a  finite  partitioning  of  the  set 
of  clock- valuations  (as  in  the  case  of  ordinary  regions),  priced  regions  also  asso¬ 
ciate  costs  to  each  individual  clock- valuation  within  the  region.  However,  as  we 
shall  see  in  the  following,  priced  regions  may  be  presented  and  manipulated  in 
a  symbolic  manner  and  are  thus  suitable  as  an  algorithmic  basis. 

Definition  3  (Priced  Regions) .  Given  set  S,  let  Seq{S)  be  the  set  of  finite 
sequences  of  elements  of  S.  A  priced  clock  region  over  a  finite  set  of  clocks  C 

R=  (h,  [ro,...  ,rfc],[co,...  ,ci]) 

is  an  element  of  (C  N)  x  Seq{2^)  x  Seq(N),  with  k  =  I,  C  —  Uig{o,... 
n  n  =  0  when  j,  and  i  >  0  implies  that  Vi  0. 

Given  a  clock  valuation  u  G  and  region  R  =  {h,[ro, . . .  ,  r^],  [cq,  . . .  ,  cjt]), 
u  G  R  iff 

1.  h  and  u  agree  on  the  integer  part  of  each  clock  in  C, 

2.  X  ^  To  iff  frac(u(n:))  =  0, 

3.  x,y  e  Ti  frac(u(a:))  =  frac(u(y)),  and 

4.  X  e  Ti,  y  e  Vj  and  i  <  j  frac('u(3;))  <  frac(u(^/)). 

For  a  priced  region  R  =  (/i,  [ro, . . .  ,rfc],  [cq,  . . .  ,Cfc])  the  first  two  components 
of  the  triple  constitute  an  ordinary  (unpriced)  region  R  =  (/i,  [ro, . . .  ,rfc]).  The 
naturals  cq,  . . .  ^Ck  are  the  costs,  which  are  associated  with  the  vertices  of  the 
closure  of  the  (unpriced)  region,  as  follows.  We  start  in  the  left-most  lower  vertex 
of  the  exterior  of  the  region  and  associate  cost  cq  with  it,  then  move  one  time 
unit  in  the  direction  of  set  to  the  next  vertex  of  the  exterior,  and  associate 
cost  Cl  with  that  vertex,  then  move  one  unit  in  the  direction  of  etc.  In  this 
way,  the  costs  cq,  . . .  ,  c^,  span  a  linear  cost  plane  on  the  fc-dimensional  unpriced 
region. 
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The  closure  of  the  unpriced  region  R  is  the  convex  hull  of  the  vertices.  Each 
clock  valuation  u  e  R  is  a  (unique)  convex  combination^  of  the  vertices.  There¬ 
fore  the  cost  of  u  can  be  defined  as  the  same  convex  combination  of  the  cost  in 
the  vertices.  This  gives  the  following  definition: 

Definition  4  (Cost  inside  Regions).  Given  priced  region  R  =  (h,  [ro, ...  ,rfc], 
[co, . . .  ,  Cfc])  and  clock  valuation  u  e  R,  the  cost  of  u  in  R  is  defined  as: 


k-i 

cost(n,  R)  =  Co  -f  ^  frac(n(a:^_i))  *  {ci^i  -  Ci) 

i=0 

where Xj  is  some  clock  in  Vj.  The  minimal  cost  associated  with  R  is  mincost(i?)  = 
min({co,...  ,Cfc}). 

In  the  symbolic  state-space,  constructed  with  the  priced  regions,  the  costs  will 
be  computed  such  that  for  each  concrete  state  in  a  symbolic  state,  the  cost 
associated  with  it  is  the  minimal  cost  for  reaching  that  state  by  the  symbolic  path 
that  was  followed.  In  this  way,  we  always  have  the  minimal  cost  of  all  concrete 
paths  represented  by  that  symbolic  path,  but  there  may  be  more  symbolic  paths 
leading  to  a  symbolic  state  in  which  the  costs  are  different.  Note  that  the  cost 
of  a  clock  valuation  in  the  region  is  computed  by  adding  fractions  of  costs  for 
equivalence  sets  of  clocks,  rather  than  for  each  clock. 

To  prepare  for  the  symbolic  semantics,  we  define  in  the  following  a  number 
of  operations  on  priced  regions.  These  operations  are  also  the  ones  used  in  the 
algorithm  for  finding  the  optimal  cost  of  reaching  a  location. 

The  delay  operation  computes  the  time  successor,  which  works  exactly  as  in 
the  classical  (unpriced)  regions.  The  changing  dimensions  of  the  regions  cause 
the  addition  or  deletion  of  vertices  and  thus  of  the  associated  cost.  The  price 


(5.1) 


[{^Uy}] 


mi 

Cq 


Cq+P 


Co 


[0,{x},{y}]  [{y},{x}] 


y  y 


[0,{x,y}l  l{x},{y}l 


y  y 


[0.{xMy}]  [{xMy}] 


Fig.  4.  Delay  and  reset  operations  for  two-dimensional  priced  regions. 


^  A  linear  expression  where  >  0. 
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argument  will  be  instantiated  to  the  price  of  the  location  in  which  time  is  passing; 
this  is  needed  only  when  a  vertex  is  added.  The  two  cases  in  the  operation  are 
illustrated  in  Fig.  4  to  the  left  (5.1)  and  (5.2). 

Definition  5  (Delay).  Given  a  priced  region  i?  =  (/i,  [ro, . . .  ,  rjt],  [cq,  . . .  ,  Ck]) 
and  a  price  p,  the  function  delay  is  defined  as  follows: 

1.  If  ro  is  not  empty,  then 

de\ay(R,p)  =  (/i,  [0,  ro, . . .  .r/t],  [co, . . .  ,Ck,co+p]) 

2.  If  ro  is  empty,  then 

delay(J?,p)  (/i  ,  [r/j,  ri, . . .  ,  r/c— i],  [ci, . . .  ,  c^]) 
where  h'  =  h  incremented  for  all  clocks  in  r^ 

When  resetting  a  clock,  a  priced  region  may  lose  a  dimension.  If  so,  the  two  costs, 
associated  with  the  vertices  that  are  collapsed,  are  compared  and  the  minimum 
is  taken  for  the  new  vertex.  Two  of  the  three  cases  in  the  operation  is  illustrated 
in  Fig.  4  to  the  right  (6.2)  and  (6.3). 

Definition  6  (Reset).  Given  a  priced  region  R  =  (/i,  [ro, . . .  ,  r^],  [cq,  . . .  ,  Ck]) 
and  a  clock  x  €  ri,  the  function  reset  is  defined  as  follows: 

1.  Ifi  =  0  then  reset(x,  R)  =  {h',  [ro, ...  ,  r^],  [cq,  . . .  ,  Ck]),  where  h'  =  h  with  x 
set  to  zero 

2.  If  i  >  0  and  r^  ^  {x},  then 

reset(x,  R)  (/i  ,  [ro  U  {3^} 5  ■  •  •  5  \  {^} 5  ■  •  •  ?  ?  [cqj  •  •  •  ?  ^fc]) 

where  h'  =  h  with  x  set  to  zero 

3.  Ifi>0  and  r^  =  {x},  then 

reset(x,R)  =  (^',  [ro  U  {x}, . . .  ,  ri_i, n+i, . . .  ,rfc], 

[cq?  •  •  *  5  —  i  —  C  ,  •  5  ^k\) 

where  c'  =  min(ck~i,Ck-i+i) 

h'  =  h  with  X  set  to  zero 

The  reset  operation  on  a  set  of  clocks:  reset(C'  U  {x},  R)  =  reset(C,  reset(x,  R)), 
and  reset(0,  R)  =  R. 

The  price  argument  in  the  increment  operation  will  be  instantiated  to  the  price 
of  the  particular  transition  taken;  all  costs  are  updated  accordingly. 

Definition  7  (Increment).  Given  a  priced  region  R  =  {h,[ro, . . .  ,rfc],  [co, . . .  , 
Ck])  and  a  price  p,  the  increment  of  R  with  respect  to  p  is  the  priced  region 
R  0  p  =  (/i,  [ro, . . .  ,  rfc],  [cj), . . .  ,  4])  where  c •  =  +  p. 
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If  in  region  R,  no  clock  has  fractional  part  0,  then  time  may  pass  in  R,  that 
is,  each  clock  valuation  in  R  has  a  time  successor  and  predecessor  in  R.  When 
changing  location  with  R,  we  must  choose  for  each  clock  valuation  um  R  between 
delaying  in  the  previous  location  until  u  is  reached,  followed  by  the  change  of 
location,  or  changing  location  immediately  and  delaying  to  u  in  the  new  location. 
This  depends  on  the  price  of  either  location.  For  this  the  following  operation  self 
is  useful. 

Definition  8  (Self).  Given  a  priced  region  R  =  (/i,  [ro,...  ,rfc],  [cq,  . . .  ,Cfc]) 
and  a  price  p,  the  function  self  is  defined  as  follows: 

1.  Ifro  is  not  empty,  then  self (R,p)  =  R. 

2.  If  ro  is  empty,  then 

se\f{R,p)  =  {h,[ro,...  ,rA;],  [cq,  . . ,  ,Cfc_i,c']) 
where  c'  =  min(cfc,  cq  4-  p) 


Definition  9  (Comparison).  Two  priced  regions  may  be  compared  only  if 
their  unpriced  versions  are  equal:  (h,  [ro, . . .  ,  r^],  [co, . . .  ,  c^])  <  (/i',  [rQ, . . .  ,  r^,], 
[cq,  iff  h  —  h',k  =  k',  and  for  0  <  i  <  k:  Vi  =  r  ■  and  Ci  <c[. 

The  operations  delay  and  self  satisfy  the  following  useful  properties: 

Proposition  1  (Interaction  Properties). 

1.  self {R,p)  <  R, 

2.  self  {self  {R,  p),  p)  ~  self  {R,p), 

3.  delay(self(i?,p),p)  <  delay{i2,p), 

4.  self(delay(i?,p),p)  =  delay(i2,p), 

5.  self{R  0  g,p)  =  self(i2,p)  0  q, 

6.  delay(i?  0  9,p)  =  delay(i?,p)  0  q, 

7.  For  g  G  B{C),  whenever  R  G  g  then  self {R,p)  G  g. 

Stated  in  terms  of  the  cost,  cost(u,R),  of  an  individual  clock  valuation,  u,  of  a 
priced  region,  R,  the  symbolic  operations  behave  as  follows: 

Proposition  2  (Cost  Relations). 

1.  Let  R  =  {h,  [ro, ...  , r^],  [cq,  . . .  ,  c^]).  If  u  G  R  and  u  +  d  G  R  then  cost(u  + 
d,  R)  —  cost{'U,  R)  d-d*  {ck  -  cq). 

2.  IfR  =  se\f{R,p),  uG  R  andu-\-d  G  delay(R,p)  then  cost  {u-\-d,  delay  {R,p))  = 
cost(w,  R)  d-  d*p. 

3.  cost(ti,  reset(a:,  R))  =  inf{  cost(u,  i?)  |  [a:  0]r  =  w  }. 
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4  Symbolic  Semantics  and  Algorithm 

In  this  section,  we  provide  a  symbolic  semantics  for  linearly  priced  timed  au¬ 
tomata  based  on  the  notion  of  priced  regions  and  the  associated  operations 
presented  in  the  previous  section.  As  a  main  result  we  shown  that  the  cost  of 
an  execution  of  the  underlying  automaton  is  captured  sufficiently  accurately. 
Finally,  we  present  an  algorithm  based  on  priced  regions.  We  refer  the  reader  to 
the  full  version  of  this  paper  for  the  proofs  not  given  here. 

Definition  10  (Symbolic  Semantics).  The  symbolic  semantics  of  a  LPTA  A 
is  defined  as  a  transition  system  with  the  state-space  L  x  ((C  N)  x  Seq{2^)  x 
Seq{N)),  with  initial  state  (Iq,  (ho,  [C],  [0]))  (where  ho  assigns  zero  to  the  integer 
part  of  all  clocks  in  C ),  and  with  the  following  transition  relation: 

-  (l,R)  (Z,delay(i?,P(0))  if  delay(i?,  P(/))  e  /(/). 

-  (l,R)  ->  {l',R')  if  there  exists  g,  r  such  that  I  I',  R  €  g,  R'  = 

reset(P,r)  eP((/,p,a,r,/'))  and  R'  €  1(1'). 

~  (/,P)^(/,self(P,P(/))) 

In  the  remainder,  states  and  executions  of  the  symbolic  transition  system  for 
LPTA  A  will  be  referred  to  as  the  symbolic  states  and  executions  of  A. 

Lemma  1.  Given  LPTA  A,  for  each  execution  a  of  A  that  ends  in  state  (l,u), 
there  is  a  symbolic  execution  (3  of  A,  that  ends  in  symbolic  state  (l,R),  such 
that  u  G  R,  and  cost(u,R)  <  cost(Q;). 


Lemma  2,  Whenever  (/,  R)  is  a  reachable  symbolic  state  and  u  G  R,  then 
mincost((i,u))  <  cost(w,P). 

Combining  the  two  lemmas  we  obtain  as  a  main  theorem  that  the  symbolic 
semantics  captures  (sufficiently)  accurately  the  cost  of  reaching  states  and  loca¬ 
tions: 

Theorem  1.  Let  I  be  a  location  of  a  LPTA  A.  Then 

mincost(/)  =  min({  mincost(P)  |  (/,  R)  is  reachable}) 


Example  2.  We  now  return  to  the  linearly  priced  timed  automaton  in  Fig.  2 
where  the  value  of  both  o:  and  f3  is  two,  and  look  at  its  symbolic  state-space. 
The  shaded  area  in  Fig.  5(i)  including  the  lines  in  and  around  the  shaded  area 
represents  some  of  the  reachable  priced  regions  in  location  B  after  time  has 
passed  (a  number  of  delay  actions  have  been  taken).  Only  priced  regions  with 
integer  values  up  to  3  are  shown.  The  numbers  are  the  cost  of  the  vertices.  The 
shaded  area  in  Fig.  5{ii)  represents  in  a  similar  way  some  of  the  reachable  priced 
regions  in  location  C  after,  time  has  passed.  □ 
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Fig.  5.  Two  reachable  sets  of  priced  regions. 


The  introduction  of  priced  regions  provides  a  first  step  towards  an  algorithmic 
solution  for  the  minimum-cost  reachability  problem.  However,  in  the  present 
form  both  the  integral  part  as  well  as  the  cost  of  vertices  of  priced  regions 
may  grow  beyond  any  given  bound  during  symbolic  exploration.  In  the  unpriced 
case,  the  growth  of  integral  parts  is  often  dealt  with  by  suitable  abstractions  of 
(unpriced)  regions,  taking  the  maximal  constant  of  the  given  timed  automaton 
into  account.  Here  we  have  chosen  a  very  similar  approach  exploiting  the  fact^ 
that  any  LPTA  A  may  be  transformed  into  an  equivalent  “bounded’  LPTA  A 
in  the  sense  that  A  and  A  reaches  the  same  locations  with  the  exact  same  cost. 

Theorem  2.  Let  A  =  {L,lo,E,I,P)  be  a  LPTA  with  maximal  constant  max. 
Then  there  exists  a  bounded  time  equivalent  of  A,  A  =  {LJq^E'^P^P'),  satis¬ 
fying  the  following: 

1.  Whenever  (/,  u)  is  reachable  in  A,  then  for  all  x  ^  C,  u{x)  <  max-f2. 

2.  For  any  location  I  e  L,  I  is  reachable  with  cost  c  in  A  if  and  only  if  I  is 
reachable  with  cost  c  in  A 

Now,  we  suggest  in  Fig.  6  a  branch-and-bound  algorithm  for  determining  the 
minimum-cost  of  reaching  a  given  target  location  Ig  from  the  initial  state  of  a 
LPTA.  All  encountered  states  are  stored  in  the  two  data  structures  Passed  and 
Waiting,  divided  into  explored  and  unexplored  states,  respectively.  The  global 
variable  COST  stores  the  lowest  cost  for  reaching  the  target  location  found  so 
far.  In  each  iteration,  a  state  is  taken  from  Waiting.  If  it  matches  the  target 
location  Ig  and  has  a  lower  cost  than  the  previously  lowest  cost  COST,  then 
Cost  is  updated.  Then,  only  if  the  state  has  not  been  previously  explored  with 
a  lower  cost  do  we  add  it  to  PASSED  and  add  the  successors  to  WAITING.  This 
bounding  of  the  search  in  line  6  of  Fig.  6  may  be  optimized  even  further  by  adding 
the  constraint  mincost(i?)  <  COST;  i.e.  we  only  need  to  continue  exploration  if 
the  minimum  cost  of  the  current  region  is  below  the  optimal  cost  computed  so 
far.  Due  to  Theorem  1,  the  algorithm  of  Fig.  6  does  indeed  yield  the  correct 
minimum-cost  value. 

Theorem  3  .  When  the  algorithm  in  Fig.  6  terminates,  the  value  of  Cost  equals 
mincost(/g). 
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Cost  :=  cxd,  Passed  :=  0,  Waiting  :=  {{lo,Ro)} 
while  Waiting  7^  0  do 

select  (/,/?)  from  Waiting 
if  I  =  Ig  and  mincost(R)  <  Cost  then 
Cost  :=  mincost(R) 

if  for  all  {I,  R')  in  Passed:  R'  R  then 
add  {I,  R)  to  Passed 

for  all  {I',  R')  such  that  {I,  R)  {I',  R'):  add  {I',  R')  to  Waiting 
return  Cost 


Fig.  6.  Branch-and-bound  state-space  exploration  algorithm. 


Proof.  First,  notice  that  if  can  reach  then  a  state  (h.R'i), 

where  R[  <  Ri,  can  reach  a  state  such  that  R2  <  R2^  We  prove  that 

Cost  equals  min{mincost(J?)  |  {lg,R)  is  reachable}.  Assume  that  this  does  not 
hold.  Then  there  exists  a  reachable  state  {lg,R)  where  mincost(i?)  <  Cost, 
Thus  the  algorithm  must  at  some  point  have  discarded  a  state  (/',  R')  on  the 
path  to  (lg,R).  This  can  only  happen  in  line  6,  but  then  there  must  exist  a 
state  (IfR")  G  Passed,  where  R'^  <  R\  encountered  in  a  prior  iteration  of 
the  loop.  Then,  there  must  be  a  state  {lg,R"')  reachable  from  and 

Cost  <  mincost(i?'")  <  mincost(i?),  contradicting  the  assumption.  The  theo¬ 
rem  now  follows  from  Theorem  1.  □ 

For  bounded  LPTA,  application  of  Higman’s  Lemma  [12]  ensures  termination. 
In  short,  Higman’s  Lemma  says  that  under  certain  conditions  the  embedding 
order  on  strings  is  a  well  quasi-order. 

Theorem  4.  The  algorithm  in  Fig.  6  terminates  for  any  bounded  LPTA. 

Proof.  Even  if  A  is  bounded  (and  hence  yields  only  finitely  many  unpriced  re¬ 
gions),  there  are  still  infinitely  many  priced  regions,  due  to  the  unbounded¬ 
ness  of  cost  of  vertices.  However,  since  all  costs  are  positive  application  of  Hig¬ 
man’s  lemma  ensures  that  one  cannot  have  an  infinite  sequence  ((c|, . . .  : 

0  <  z  <  00)  of  cost-vectors  (for  any  fixed  length  m)  without  <  cf  for  all 
/  =  1,...  ,7n  for  some  j  <  k.  Consequently,  due  to  the  finiteness  of  the  sets 
of  locations  and  unpriced  regions,  it  follows  that  one  cannot  have  an  infinite 
sequence  {{k^Ri)  :  0  <  z  <  00)  of  symbolic  states  without  Ij  —  and  Rj  <  R^ 
for  some  j  <  fc,  thus  ensuring  termination  of  the  algorithm.  □ 

Finally,  combining  Theorem  3  and  4,  it  follows,  due  to  Theorem  2,  that  the 
minimum-cost  reachability  problem  is  decidable. 

Theorem  5.  The  minimum-cost  problem  for  LPTA  is  decidable. 

5  Conclusion 

In  this  paper,  we  have  successfully  extended  the  work  on  regions  and  their  op¬ 
erations  to  a  setting  of  timed  automata  with  linear  prices  on  both  transitions 
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and  locations.  We  have  given  the  principle  basis  of  a  branch- and-bound  algo¬ 
rithm  for  the  minimum-cost  reachability  problem,  which  is  based  on  an  accurate 
symbolic  semantics  of  timed  automata  with  linear  prices,  and  thus  showing  the 
minimum-cost  reachability  problem  to  be  decidable. 

The  algorithm  is  guaranteed  to  be  rather  inefficient  and  highly  sensitive  to 
the  size  of  constants  used  in  the  guards  of  the  automata  —  a  characteristic 
inherited  from  the  time  regions  used  in  the  basic  data-structure  of  the  algorithm. 
An  obvious  continuation  of  this  work  is  therefore  to  investigate  if  other  more  (in 
practice)  efficient  data  structures  can  be  found.  Possible  candidates  include  data 
structures  used  in  reachability  algorithms  of  timed  automata,  such  as  DBMs, 
extended  with  costs  on  the  vertices  of  the  represented  zones  (i.e.  convex  sets  of 
clock  assignments).  In  contrast  to  the  priced  extension  of  regions,  operations  on 
such  a  notion  of  priced  zones®  can  not  be  obtained  as  direct  extensions  of  the 
corresponding  operations  on  zones  with  suitable  manipulation  of  cost  of  vertices. 

The  need  for  infimum  in  the  definition  of  minimum  cost  executions  arises 
from  linearly  priced  timed  automata  with  strict  bounds  in  the  guards,  such  as 
the  one  shown  in  Fig.  3  and  discussed  in  Example  1.  Due  to  the  use  of  infimum, 
a  linearly  priced  timed  automaton  is  not  always  able  to  realize  an  execution 
with  the  exact  minimum  cost  of  the  automata,  but  will  be  able  to  realize  one 
with  a  cost  (infinitesimally)  close  to  the  minimum  value.  If  all  guards  include 
only  non-strict  bounds,  the  minimum  cost  trace  can  always  be  realized  by  the 
automaton.  This  fact  can  be  shown  by  defining  the  minimum-cost  problem  for 
executions  covered  by  a  given  symbolic  trace  as  a  linear  programming  problem. 

In  this  paper  we  have  presented  an  algorithm  for  computing  minimum-costs 
for  reachability  of  linearly  priced  timed  automata,  where  prices  are  given  as 
constants  (natural  numbers).  However,  a  slight  modification  of  our  algorithm 
provides  an  extension  to  a  parameterized  setting,  in  which  (some)  prices  may  be 
parameters.  In  this  setting,  costs  within  priced  regions  will  be  finite  collections, 
C,  of  linear  expressions  over  the  given  parameters  rather  than  simple  natural 
numbers.  Intuitively,  C  denotes  for  any  given  instantiation  of  the  parameters  the 
minimum  of  the  concrete  values  denoted  by  the  linear  expressions  within  C.  Now, 
two  cost-expressions  may  be  compared  simply  by  comparing  the  sizes  of  corre¬ 
sponding  parameters,  and  two  collections  C  and  D  (both  denoting  minimums) 
are  related  if  for  any  element  of  D  there  is  a  smaller  element  in  C.  In  the  mod¬ 
ified  version  of  algorithm  Fig.  6,  Cost  will  similarly  be  a  collection  of  (linear) 
cost-expressions  with  which  the  goal-location  has  been  reached  (so  far).  From 
recent  results  in  [1]  (generalizing  Higman’s  lemma)  it  follows  that  the  ordering 
on  (parameterized)  symbolic  states  is  again  a  well-quasi  ordering,  hence  guaran¬ 
teeing  termination  of  our  algorithm.  Also,  we  are  currently  working  on  extending 
the  algorithmic  solution  offered  here  to  synthesis  of  minimum-cost  controllers  in 
the  sense  of  [4].  In  this  extension,  a  priced  region  will  be  given  by  a  conven¬ 
tional  unpriced  region  together  with  a  min-max  expression  over  cost  vectors  for 
the  vertices  of  the  region.  In  both  the  parametric  and  the  controller  synthesis 
case,  it  follows  from  recent  results  in  [1]  (generalizing  Higman’s  lemma)  that  the 


®  In  particular,  the  reset-operation. 
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orderings  on  symbolic  states  are  again  well-quasi  orderings,  hence  guaranteeing 
termination  of  our  algorithms. 
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Abstract.  In  this  paper  we  describe  a  hybrid  model  and  an 
optimization-based  control  strategy  for  solving  a  traction  control  prob¬ 
lem  currently  under  investigation  at  Ford  Research  Laboratories.  We 
show  through  simulations  on  a  model  and  a  realistic  set  of  parameters 
that  good  and  robust  performance  is  achieved.  Furthermore,  the  result¬ 
ing  optimal  controller  is  a  piecewise  linear  function  of  the  measurements 
that  can  be  implemented  on  low  cost  control  hardware. 


1  Introduction 

For  more  than  a  decade  advanced  mechatronic  systems  controlling  some  aspects 
of  vehicle  dynamics  have  been  investigated  and  implemented  in  production  [13]. 
Among  them,  the  class  of  traction  control  problems  is  one  of  the  most  studied. 
Traction  controllers  are  used  to  improve  a  driver’s  ability  to  control  a  vehicle 
under  adverse  external  conditions  such  as  wet  or  icy  roads.  By  maximizing  the 
tractive  force  between  the  vehicle’s  tire  and  the  road,  a  traction  controller  pre¬ 
vents  the  wheel  from  slipping  and  at  the  same  time  improves  vehicle  stability 
and  steerability.  In  most  control  schemes  the  wheel  slip,  i.e.,  the  difference  be¬ 
tween  the  normalized  vehicle  speed  and  the  speed  of  the  wheel  is  chosen  as  the 
controlled  variable.  The  objective  of  the  controller  is  to  maximize  the  tractive 
torque  while  preserving  the  stability  of  the  system.  The  relation  between  the 
tractive  force  and  the  wheel  slip  is  nonlinear  and  is  a  function  of  the  road  condi¬ 
tion  [2].  Therefore,  the  overall  control  scheme  is  composed  of  two  parts:  a  device 
that  estimates  the  road  surface  condition,  and  a  traction  controller  that  regulates 
the  wheel  slip  at  any  desired  value.  Regarding  the  second  part,  several  control 
strategies  have  been  proposed  in  the  literature  mainly  based  on  sliding-mode  con¬ 
trollers,  fuzzy  logic  and  adaptive  schemes  [5,14,4,19,20,17,2,18].  Such  control 
schemes  are  motivated  by  the  fact  that  the  system  is  nonlinear  and  uncertain. 

The  presence  of  nonlinearities  and  constraints  on  one  hand,  and  the  sim¬ 
plicity  needed  for  real-time  implementation  on  the  other,  have  discouraged  the 
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Fig.  1.  Simple  vehicle  model 


design  of  optimal  control  strategies  for  this  kind  of  problem.  Recently  we  pro¬ 
posed  a  new  framework  for  modeling  hybrid  systems  [8]  and  an  algorithm  to 
synthesize  piecewise  linear  optimal  controllers  for  such  systems  [6] .  In  this  paper 
we  describe  how  the  hybrid  framework  [8]  and  the  optimization-based  control 
strategy  [6]  can  be  successfully  applied  for  solving  the  traction  control  problem 
in  a  systematic  way.  We  show,  through  simulations  on  a  simplified  model  and  for 
a  set  of  parameters  provided  by  Ford  Research  Laboratories,  that  good  and  ro¬ 
bust  performance  can  be  achieved.  Furthermore,  the  resulting  optimal  controller 
consists  of  a  piecewise  linear  function  of  the  measurements,  that  can  be  easily 
implemented. 

A  mathematical  model  of  the  vehicle/tire  system  is  introduced  in  Section  2. 
The  hybrid  modeling  and  the  optimal  control  strategy  are  discussed  in  Sec¬ 
tions  2.1  and  3,  respectively.  In  Section  4  we  derive  the  piecewise  affine  optimal 
control  law  for  traction  control  and  present  some  simulation  results. 


2  Vehicle  Model 


The  model  of  the  vehicle  used  for  the  design  of  the  traction  controller  is  depicted 
in  Figure  1,  and  consists  of  the  equations 


\'^v  J  \  0  0  J  \Vv  J  \0  J 


Tc  + 


(1) 


with 


fc(t)  =  -kiTcit)  +  kiTd{t  -  Tf) 


(2) 


where  the  involved  physical  quantities  and  parameters  are  described  in  Table  1. 
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Table  1.  Physical  quantities  and  parameters  of  the  vehicle  model 


Engine  speed 

rt 

Tire  radius 

lES 

Vehicle  speed 

Actual  combustion  torque 

im 

Combined  engine/wheel  inertia 

Td 

Desired  combustion  torque 

lESI 

Engine  damping 

Frictional  torque  on  the  tire 

\WM 

Total  driveline  gear  ratio  between  We  and  Vv 

JL 

Road  coefficient  of  friction 

I^BI 

Vehicle  mass 

Tf 

Fueling  to  combustion  pure  delay  period 

lEQII 

Wheel  slip 

_ 

The  frictional  torque  Tt  is  approximated  as  a  piecewise  linear  function  of  the 
slip  Auj  and  of  the  road  coefficient  of  friction  ^ 


k\Aijj  if  Alj  <  A(jjI 
k^Auj  if  Au)  >  AljI 


for  fJii<  }x<  z  =  0, . . .  ,  AT 


(3) 


as  depicted  in  Figure  2(a). 

Model  (1)  contains  two  states  for  the  mechanical  system  downstream  of  the 
manifold/fueling  dynamics.  The  first  equation  represents  the  wheel  dynamics 
under  the  effect  of  the  combustion  torque  and  of  the  traction  torque,  while 
the  second  one  describes  the  longitudinal  motion  dynamics  of  the  vehicle.  In 
addition  to  the  mechanical  equations  (1)  the  air  intake  and  fueling  model  (2) 
also  contributes  to  the  dynamic  behaviour  of  the  overall  system.  For  simplicity, 
the  intake  manifold  dynamics  is  modeled  as  a  first  order  filter  and  the  fueling 
combustion  delay  is  modeled  as  a  pure  delay. 


2.1  Discrete-Time  Hybrid  Model 

Hybrid  systems  provide  a  unified  framework  for  describing  processes  evolving  ac¬ 
cording  to  continuous  dynamics,  discrete  dynamics,  and  logic  rules  [1,16,10,3]. 
The  interest  in  hybrid  systems  is  mainly  motivated  by  the  large  variety  of  prac¬ 
tical  situations,  for  instance  real-time  systems,  where  physical  processes  interact 
with  digital  controllers.  Several  modeling  formalisms  have  been  developed  to  de¬ 
scribe  hybrid  systems  [12,15],  among  them  the  class  of  Mixed  Logical  Dynamical 
(MLD)  systems  introduced  by  Bemporad  and  Morari  [8].  Examples  of  real-world 
applications  that  can  be  naturally  modeled  within  the  MLD  framework  are  re¬ 
ported  in  [7,8,9].  The  language  HYSDEL  (HYbrid  Systems  DEscription  Lan¬ 
guage)  was  developed  in  [21]  to  obtain  MLD  models  from  of  a  high  level  textual 
description  of  the  hybrid  dynamics. 

The  model  obtained  in  Section  2  is  transformed  into  an  equivalent  discrete¬ 
time  MLD  model  through  the  following  steps: 

1.  Discretize  the  model  {l)-(3)  with  sampling  time  Ts  =  20  ms; 

2.  Introduce  an  auxiliary  logic  variable  5^  for  each  interval  [yUi,/ii^i]  whose 
value  can  be  1  or  0  depending  on  the  value  of  the  slip  Alj^  as  shown  in 
Figure  2(b). 
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(a)  Pull  model 


(b)  Piecewise  linear  model  of  the 
tire  torque  n  with  ^  (^j,/Xi+i) 


Fig.  2.  Model  of  the  tire  torque  rt  as  a  function  of  the  slip  Alj  and  road  coefficient 
adhesion  fx 


Remark  1.  In  the  sequel  we  will  use  a  simplified  model  where  the  slopes  k\  = 
kf  =  ...  =  k^  and  k^  =  k^  =  . . .  =  k^ ,  while  the  breakpoints  cjJ  in  (3)  are 
allowed  to  be  different.  In  this  case  the  number  of  auxiliary  logic  variables  5^ 
reduces  from  log2  AT  to  1,  at  the  price  of  a  “rougher”  model  of  the  nonlinearity. 

The  resultant  MLD  system  is  the  following^: 

x{t  +  1)  =  Ax{t)  +  Biu{t)  +  +  Bsz{t)  (4a) 

y{t)  =  Cx{t)  +  Diu{t)  +  D2Sit)  +  D3z{t)  (4b) 

E2S{t)  +  E3z{t)  <  Eiu{t)  +  E4x{t)  +  Es  (4c) 

where  x  £  (xi  =  AuJd,  X2  =  xs  —  Vy,  X4  =  Tt,  X5  —  Tc),  u  €  R,  (u  =  r^), 

y  e  R  (y  =  Auj),  S  e  {0,1}  and  z  €  R^.  The  variables  S  and  z  are  auxiliary 

variables  whose  value  is  determined  uniquely  by  the  inequalities  (4c)  once  x(t) 
and  u(t)  are  fixed  [8]. 

In  Figure  4  we  compare  the  evolution  of  the  discrete-time  MLD  model  (4) 
with  the  evolution  of  the  continuous  time  model  (l)-(3),  depicted  in  Figure  3, 
when  /i  =  .1,  AcJb  =  2  rad/s  and  a  pulse  torque  =  50  Nm  is  applied  to  the 
system.  The  MLD  model  (4)  captures  in  discrete  time  the  hybrid  behavior  of 
the  system  satisfactorily. 


3  Optimal  Control 

It  is  clear  from  Figure  2(b)  that  if  the  slip  increases  beyond  Aw^y  the  driving  force 
on  the  tire  decreases  considerably  and  the  vehicle  cannot  speed  up  as  desired.  By 


^  The  numerical  values  of  the  matrices  in  (4)  are  reported  in  the  Appendix. 
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Fig.  3.  Simulink  scheme  of  the  vehicle  model 


Fig.  4.  Continuous  time  simulation  of  the  Matlab-Simulink  block  in  Figure  3  (solid 
line),  discrete- time  simulation  of  the  MLD  model  (dashed  line) 


maximizing  the  tractive  force  between  the  vehicle’s  tire  and  the  road,  a  traction 
controller  prevents  the  wheel  from  slipping  and  at  the  same  time  improves  vehicle 
stability  and  steerability  The  overall  control  scheme  is  composed  of  two  parts:  a 
device  that  estimates  the  road  surface  condition,  and  a  traction  controller  that 
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regulates  the  wheel  slip  at  any  desired  value.  In  this  paper  we  will  focus  only  on 
the  second  part,  as  the  first  one  is  available  at  Ford  Research  Laboratories. 

Once  the  road  coefficient  of  adhesion  (jl  has  been  estimated,  a  desired  wheel 
sleep  is  chosen  corresponding  to  the  breakpoints  Aujj^  in  model  (3), 

(JL  6  [(ik-i, P'k]-,  where  the  frictional  torque  Tt{Auji)  on  the  tire  is  maximized. 
Alternatively,  to  increase  the  safety  of  the  controller  [18]  we  could  avoid  operat¬ 
ing  in  the  region  where  the  slope  of  the  curve  Tt{Auj)  is  negative,  see  Figure  2(b), 
by  simply  choosing  Au}^{(i)  <  Auj^  {(jl  C  [(ik-h (^k])-  The  control  system  takes 
the  desired  wheel  slip  Au^  and  measured  wheel  speed  as  input  and  generates  the 
desired  engine  torque.  The  following  constraints  on  the  torque  and  its  variation 
need  to  be  satisfied: 


20  Nm  <Td<  176  Nm 
'^d{i)  ^  2000  Nm/s 


(5) 

(6) 


In  the  sequel  we  describe  how  a  Model  Predictive  Controller  (MPC)  can 
be  designed  for  the  posed  traction  control  problem  described.  The  main  idea 
of  MPC  is  to  use  the  model  of  the  plant  to  predict  the  future  evolution  of  the 
system.  Based  on  this  prediction,  at  each  time  step  t  a  certain  performance 
index  is  optimized  under  operating  constraints  with  respect  to  a  sequence  of 
future  input  moves.  The  first  of  such  optimal  moves  is  the  control  action  applied 
to  the  plant  at  time  t.  At  time  t  -h  1,  a  new  optimization  is  solved  over  a  shifted 
prediction  horizon.  For  the  traction  control  problem,  at  each  time  step  t  the 
following  finite  horizon  optimal  control  problem  is  solved: 


T-l 


min 

=0 

(  MLD  dynamics  (4) 


[AuI 


subj.  to  < 


At, 


min 

^min 


<  U{t  +  /c)  <  Tmax,  A:  =  0,  1,  .  .  .  ,T  -  1 

<  Su{t  -\-k)  <  ATmax,  A:  =  0, 1, , . .  ,  r  -  1 

<  x{t  -h  k\t)  <  Xmax,  A:  =  1, . . .  ,  T  -  1 


(7) 


(8) 


where  Au^"^  =  {Su{t), . . .  ,Su{t-\-T  -1)},  and  “(t  +  A:|t)”  denotes  the  predicted 
value  at  time  t  +  k  based  on  the  state  information  available  at  time  t.  Note 
that  the  optimization  variables  are  not  the  future  inputs  Ut^k  ?  but  the  variation 
6u{t  +  A:)  =  u{t  +  A:)  —  u{t  +  A:  —  1),  which  makes  it  necessary  to  increase  the 
dimension  of  the  state  vector  by  one  to  include  the  previous  torque  Td{t  —  \)  as 
a  an  additional  state  XQ[t)  =  Td{t  —  1). 

Problem  (7)-(8)  can  be  translated  into  a  mixed  integer  linear  program 
(MILP)  (the  minimization  of  a  linear  cost  function  subject  to  linear  constraints 
where  variables  can  be  binary  and/or  continuous)  of  the  form: 

min  /f  2e  +  fd^d 

Z  =  {Zc,Zd} 

subj.  to  GcZc  +  GcZd  <  S  Fx{t) 
where  2:c  6  and  Zd  6  {0, 1}"^. 


(9) 
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Given  the  measurement  of  the  state  x{t),  problem  (9)  is  solved  at  each  time 
step,  but  only  the  first  optimal  input  u*{t)  —  rd(t  -  1)  +  5uq  is  implemented  as 
the  new  command  torque  rd{t).  At  the  next  time  step  the  procedure  is  repeated 
starting  with  the  new  measurement  of  the  state. 

The  design  of  the  controller  is  performed  in  two  steps.  First,  the  MFC  con¬ 
troller  (7)-(8)  based  on  model  (4)  is  tuned  in  simulation  until  the  desired  per¬ 
formance  is  achieved.  The  MFC  controller  is  not  directly  implement  able,  as  it 
would  require  the  MILF  (9)  to  be  solved  on-line,  which  is  clearly  prohibitive 
on  standard  automotive  control  hardware.  Therefore,  for  implementation,  in  the 
second  phase  the  explicit  piecewise  linear  form  of  the  MFC  law  (see  Section  4.2) 
is  computed  off-line  by  using  the  multi-parametric  mixed  integer  programming 
solver  presented  in  [11].  Although  the  resulting  piecewise  linear  control  action 
is  identical  to  the  MFC  designed  in  the  first  phase,  the  on-line  complexity  is 
reduced  to  the  simple  evaluation  of  a  piecewise  linear  function. 

4  Controller  Design 

The  only  parameter  of  the  controller  (7)-(8)  to  be  tuned  is  the  horizon  length  T. 
By  increasing  the  prediction  horizon  the  controller  performance  improves,  but 
at  the  same  time  the  number  of  constraints  in  (8)  increases.  As  will  be  explained 
in  Section  4.2  the  complexity  of  the  final  piecewise  linear  controller  increases 
with  the  number  of  constraints  in  (8).  Therefore,  tuning  T  amounts  to  finding 
the  smallest  T  which  leads  to  a  satisfactory  closed- loop  behaviour. 


4.1  Simulations 

We  simulate  the  closed-loop  composed  of  the  traction  controller  (7)- (8)  and 
model  (l)-(2),  where  the  piecewise  linear  function  modeling  the  frictional  torque 
on  the  tire  Tt  (3)  is  replaced  by  a  more  accurate  nonlinear  model  provided  by 
Ford,  see  Figure  5.  The  actual  combustion  torque  Tc  is  estimated  from  the  two 
measurements  Wg  and  by  using  an  extended  Kalman  Filter  designed  for  the 
FWA  model. 

The  controlled  system  is  simulated  with  an  initial  vehicle  speed  of  zero.  The 
intake  manifold  state  Tg  is  set  to  a  large  torque  value,  namely  rc(0)  —  100  Nm,  in 
order  to  approximate  a  wide-open  throttle  launch  from  a  standstill.  In  Figure  6 
we  simulate  a  straight-ahead  driving  with  a  transition  at  time  t*  =  2  s  from  a 
high  coefficient  of  friction  //  =  0.9,  and  AiVd  —  18  rad/s  (cement  pavement)  to 
a  low  one  {i  —  0.1,  Aujd  =  2  rad/s  (dry  ice).  The  simulations  show  the  good 
performance  of  the  controller  despite  the  large  mismatch  between  the  nonlinear 
model  of  the  frictional  torque  model  and  the  piecewise  linearized  one. 

The  following  controllers  are  simulated: 

-  Controller  1  (Figure  6(a)):  T=3; 

-  Controller  2  (Figure  6(b)):  T=9; 

The  Simulink  control  diagram  used  for  simulation  is  shown  in  Figure  5. 


A  Hybrid  Approach  to  Traction  Control 


169 


x’  =  Ax+Bu 
y  =  Cx+Du 


Delay 


Manifold 

Transfer  Function 


x'  =  Ax+Bu 
y  =  Cx+Du 


03„ 


MATLAB 

Function 


State-Space  Model 


p  (real) 


non-linear  tire  mode! 


fx  (estimated) 
© 


Fig.  5.  Simulink  diagram  of  the  closed-loop  control  system 


4.2  Explicit  Controller 

Once  the  controller  has  been  tuned,  the  explicit  piecewise  linear  form  of  the 
MFC  law  is  computed  off-line  by  using  a  multiparametric  mixed  integer  linear 
programming  (mp-MILP)  solver,  according  to  the  approach  of  [6].  Rather  than 
solving  the  MILP  (9)  on-line  for  the  given  current  state  x{t),  the  idea  is  to  use 
the  mp-MILP  solver  to  compute  off-line  the  solution  of  the  MILP  (9)  for  all  the 
states  x{t)  within  a  given  polyhedral  set. 

As  shown  in  [6],  the  explicit  solution  z*{x(t))  of  (9)  is  a  piecewise  affine  func¬ 
tion  of  Therefore,  the  model  predictive  controller  is  also  available  explicitly, 
as  the  optimal  input  6u{t)  consists  simply  of  a  component  of  z*(x{t)).  As  a  re¬ 
sult,  the  state  space  is  partitioned  into  polyhedral  sets,  where  an  affine  control 
law  is  defined  in  each  polyhedron. 

We  remark  that  for  any  given  state  x(t)  the  on-line  solution  of  MPC  and 
the  explicit  off-line  solution  provide  the  same  result.  Therefore,  a  good  design 
strategy  consists  of  tuning  the  MPC  controller  using  simulation  and  on-line 
optimization,  and  then  to  convert  the  controller  to  its  piecewise  affine  explicit 
form.  The  explicit  controller  will  behave  in  exactly  the  same  way  at  much  lower 
computation  cost. 

The  result  of  the  mp-MILP  solver  is  a  list  of  N  records.  The  i-th  record 
contains  the  constraints  defining  the  i-th  polyhedral  region  H{i)x  <  K{i), 
H{i)  ©  and  the  corresponding  i-th  gain  6u  =  F{i)x  4-  G{i).  The  con¬ 

trol  law  can  be  implemented  on-line  in  the  following  simple  way:  (1)  determine 
the  i-th  region  that  contains  the  actual  vector  state  x{t)  (measured  and/or  es¬ 
timated);  (2)  compute  5u{t)  =  F{i)x{t)  +  G(i),  according  to  the  corresponding 
i-th  control  law. 
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(a)  Controller  1  (T=3) 


(b)  Controller  2  (T=9) 


Fig.  6-  Closed-loop  simulation  of  Controller  1  and  Controller  2.  Straight-ahead  driving 
with  a  transition  at  time  =  2  s  from  a  high  coefficient  of  friction  fj,  =  0.9,  and 
Aud  =  18  rad/s  (cement  pavement)  to  a  low  one  /Li  =  0.1,  Aojd  —  2  rad/s  (dry  ice) 


In  Figure  6(a)  we  report  the  performance  achieved  with  two  explicit 
MFC  controllers,  obtained  by  solving  the  mp-MILP  problem  for  the  box 
^min  ^  ^(0  —  ^max:  ^min  ~  [0,0, 0,-20, -20, -40]  and  = 

[20,150,10,100,300, 40]: 


—  Controller  1  :  T=3,  Number  of  regions  N  —  76,  maximum  number  of  con¬ 
straints  per  region  M  —  maxi=i,...  ,jv  =  13; 

—  Controller  2:  T=9,  Number  of  regions  N  =  243,  maximum  number  of  con¬ 
straints  per  region  M  =  25, 
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As  an  example,  we  report  only  the  first  and  last  region  of  Controller  1: 


(  —  40.0000 


r  0.0 

0.0 

-0.0 

0.0 

0.0 

-0.05-1 

r-l.O 

0.0 

0.01 

0.0 

0.0 

0.0 

0.0 

1.0 

0.0 

0.0 

0.0 

0.01 

0.0 

0.0 

1.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.02 

1.0 

-1.0 

0.0 

0.0 

0.0 

0.0 

0.0 

1.0 

0.0 

-1.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

-1.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

-0.05 

0.0 

0.0 

X  S 

1.0 

-6.13 

0.47 

-22.70 

-0.02 

0.14 

0.02 

1.0 

0.0 

0.0 

-0.0 

0.0 

0.0 

-0.05 

-1.0 

15.83 

-1.22 

58.65 

0.03 

-0.24 

-0.03 

1.0 

-8.0 

1.23 

-59.26 

-0.01 

0.07 

0.0 

0.0 

-31.57 

2.43 

-116.94 

-0.05 

0.47 

0.00 

—  1.0 

L  12.25 

-0.94 

45.38 

0.03 

-0.26 

-0.02  J 

-  — 1.0  - 

(Region  #1) 


=  < 


(10) 


r  368.11  T 
-28.34 
1363.38 
0.85 
-7.13 

L  -1.00  J 


T 


X  4-  11-59 


r  43.7456 

-3.3676 

162.0209 

0.1011  -0.8468 

-0.0  1 

r  1-0  ~\ 

-7.8681 

0.6057 

-29.1410 

-0.0153  0.1391 

0.0178 

-1.0 

-6.1306 

0.4719 

-22.7058 

-0.0180  0.1364 

0.0215 

1.0 

-8.0 

1.2317 

-59.2593 

-0.0068  0.0697 

-0.0 

0.0 

0.0 

0.0067 

0.0 

0.0  0.0 

0.0 

1.0 

0.0 

0.0 

0.0 

0.0100  0.0 

0.0 

X  < 

1.0 

-1.0 

0.0 

0.0 

0.0  0.0 

0.0 

1.0 

0.0 

-1.0 

0.0 

0.0  0.0 

0.0 

0.0 

0.0 

0.0 

-1.0 

0.0  0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

-0.0500  0.0 

0.0 

1.0 

.  0.0 

0.0 

0.0 

0.0  0.0 

-0.05  . 

L  1.0  J 

(Region  #76) 


In  Figure  7  a  zoomed  section  of  the  control  law  associated  with  Controller  1 
is  shown.  The  section  is  obtained  by  fixing  the  torque  Tc  =  20,  the  desired  slip 
AuJd  —  2,  the  friction  torque  r*  =  80,  and  the  previous  input  rd{t  —  \)  —  20.  Note 
that  the  southeast  corner  is  not  feasible  because  it  corresponds  to  a  negative  slip. 


5  Conclusion 


In  this  paper  we  described  a  hybrid  model  and  an  optimization-based  control 
strategy  for  a  traction  control  problem.  We  showed,  through  simulations  on  a 
model  and  a  realistic  set  of  parameters  from  Ford  Research  Laboratories,  that 
good  and  robust  performance  is  achieved.  Furthermore,  the  resulting  optimal 
controller  is  a  piecewise  linear  function  of  the  measurements  that  can  be  easily 
implemented  on  low  cost  hardware.  In  order  to  ease  the  implementation  of  the 
controller,  the  number  of  regions  in  the  piecewise  linear  law  should  be  reduced. 
One  possible  way  is  to  exploit  reachability  analysis  for  hybrid  systems  in  order  to 
remove  regions  which  are  never  entered,  for  all  the  operating  conditions  within 
a  realistic  set.  At  the  same  time,  for  complex  piecewise  linear  partitions,  we 
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are  developing  efficient  forms  of  implementation  that  greatly  reduce  the  num¬ 
ber  of  regions  to  be  stored  by  exploiting  properties  of  multiparametric  linear 
programming. 


Acknowledgments.  We  thank  Manfred  Morari  for  fruitful  discussions  and  his 
helpful  comments  on  the  original  manuscript. 

6  Appendix 

Below  we  report  the  numerical  values  of  the  matrices  in  (4)  obtained  by  using  the 
tool  HYSDEL.  See  http://www.aut.ee.ethz.ch/~hybrid/FordExample.html 
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Abstract.  We  consider  the  synthesis  of  optimal  controls  for  continuous 
feedback  systems  by  recasting  the  problem  to  a  hybrid  optimal  control 
problem  which  is  to  synthesize  optimal  enabling  conditions  for  switching 
between  locations  in  which  the  control  is  constant.  We  provide  a  single¬ 
pass  algorithm  to  solve  the  dynamic  programming  problem  that  arises, 
with  added  constraints  to  ensure  non-Zeno  trajectories. 


1  Introduction 

In  this  paper  we  continue  our  investigation  of  the  application  of  hybrid  systems 
and  bisimulation  to  optimal  control  problems.  In  the  first  paper  [2]  we  devel¬ 
oped  a  discrete  method  for  solving  an  optimal  control  problem  based  on  hybrid 
systems  and  bisimulation.  We  showed  that  the  value  function  of  the  discrete 
problem  converges  to  the  value  function  of  the  continuous  problem  as  a  dis¬ 
cretization  parameter  6  tends  to  zero.  In  this  paper  we  focus  on  the  pragmatic 
question  of  how  the  discretized  problem  can  be  efficiently  solved. 

Following  the  introduction  of  the  concept  of  viscosity  solution  [10,4], 
Capuzzo-Dolcetta  [3]  introduced  a  method  for  obtaining  approximations  of  vis¬ 
cosity  solutions  based  on  time  discretization  of  the  Hamilton-Jacobi-Bellman 
(HJB)  equation.  The  approximations  of  the  value  function  correspond  to  a  dis¬ 
crete  time  optimal  control  problem,  for  which  an  optimal  control  can  be  syn¬ 
thesized  that  is  piecewise  constant.  Finite  difference  approximations  were  also 
introduced  in  [5]  and  [13].  In  general,  the  time  discretized  approximation  of  the 
HJB  equation  is  solved  by  finite  element  methods.  Gonzales  and  Rofman  [9] 
introduced  a  discrete  approximation  by  triangulating  the  domain  of  the  finite 
horizon  problem  they  considered,  while  the  admissible  control  set  is  approxi¬ 
mated  by  a  finite  set.  Gonzales  and  Rofman’s  approach  is  adapted  in  several 
papers,  including  [7].  The  approach  of  [14]  uses  the  special  structure  of  an  op¬ 
timal  control  problem  to  obtain  a  single-pass  algorithm  to  solve  the  discrete 
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problem,  thus  bypassing  the  expensive  iterations  of  a  finite  element  method. 
The  essential  property  needed  to  find  a  single  pass  algorithm  is  to  obtain  a  par¬ 
tition  of  the  domain  so  that  the  cost-to-go  value  from  any  equivalence  class  of 
the  partition  is  determined  from  knowledge  of  the  cost-to-go  from  those  equiv¬ 
alence  classes  with  strictly  smaller  cost-to-go  values.  In  this  paper  we  obtain  a 
partition  of  the  domain  provided  by  a  bisimulation  partition.  The  combination 
of  the  structure  of  the  bisimulation  partition  and  the  requirement  of  non-Zeno 
trajectories  enables  us  reproduce  the  essential  property  of  [Ij],  so  that  we  obtain 
a  Dijkstra-like  algorithmic  solution.  Our  approach  has  complexity  0{N  log  N) 
if  suitable  data  structures  are  used,  where  N  is  the  number  of  locations  of  the 
finite  automaton. 

While  the  objective  is  to  solve  a  continuous  optimal  control  problem,  the 
method  can  be  adapted  to  solve  directly  the  problem  of  optimal  synthesis  of 
enabling  conditions  for  hybrid  systems.  In  that  spirit,  [11]  investigates  games  on 
timed  automata  and  obtains  a  dynamic  programming  formulation  as  well. 


2  Optimal  Control  Problem 


cl  (A)  denotes  the  closure  of  set  A.  ||  •  ||  denotes  the  Euclidean  norm.  «T(IR”) 
denotes  the  sets  of  smooth  vector  fields  on  IR”.  (j)t{x{),}i)  denotes  the  trajectory 
of  i  =  f{x,p)  starting  fi:om  2:0  and  using  control  //(•). 

Let  C/  be  a  compact  subset  of  IR”^,  i?  an  open,  bounded,  connected  subset 
of  IR’^,  and  Qf  a  compact  subset  of  O.  Define  Um  to  be  the  set  of  measurable 
functions  mapping  [0,  T]  to  U.  We  define  the  minimum  hitting  time  T  :  IR”  x 
Um  IR+  by 


00  if  {t  I  (j)t{x,p)  G  %  }  =  0 

min{t  I  (ptix^fj.)  G  i?/}  otherwise. 


(1) 


A  control  p  e  Um  specified  on  [0,T]  is  admissible  for  x  G  1?  if  (j)t{x,p)  G  Q  for 
all  t  G  [0,  Tj.  The  set  of  admissible  controls  for  x  is  denoted  Let  77. :—  {  x  G 
I  3/i  G  Ux-  T{x,p)  <  00  }.  We  consider  the  following  optimal  control  problem. 
Given  ?/  G  1?, 


rT{y,^) 

minimize  J{y,p)=  L{x{s),  p{s))ds  +  h{x{T{y,  p)))  (2) 

Jo 

subject  to  x==  f{x,  p),  a.e.  t  G  [0,  T{y,  p)]  (3) 

x(0)  =  y  (4) 

among  all  admissible  controls  p  ^  Uy.  J  :  x  Um  ^  JR  is  the  cost-to-go 

function,  h  :  IR”  — >  IR  is  the  terminal  cost,  and  L  :  IR^  x  IR”^  — IR  is  the 
instantaneous  cost  At  T(y,p)  the  terminal  cost  h{x{T{y,  p)))  is  incurred  and 
the  dynamics  are  stopped.  The  control  objective  is  to  reach  i?/  from  y  ^  Q  with 
minimum  cost. 

The  value  function  or  optimal  cost-to-go  function  V  :  IR’^  — >  IR  is  given  by 
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for  y  G  Q\Qf,  and  by  V{y)  ~  h{y)  for  y  G  Qf.  V  satisfies  the  Hamilton- Jacobi- 
Bellman  equation 

-inl{L{x,u)  +  ^f{x,u)}=0  (5) 

at  each  point  of  Tl  at  which  it  is  diflterentiable.  The  HJB  equation  is  an  infinites¬ 
imal  version  of  the  equivalent  Dynamic  Programming  Principle  (DPP)  which 
says  that 

V{x)  i  €  i? \  % 

V{x)  —  h{x)  X  G  i?/. 

Because  the  HJB  equation  may  not  have  a  solution  it  has  not  been  possible 
to  obtain  a  rigorous  foundation  for  solutions  in  the  usual  sense.  The  correct 
concept  for  solutions  is  that  of  viscosity  solutions  [10,4],  which  provide  the  unique 
solution  of  (5)  without  differentiability.  We  showed  in  [2]  that  under  assumptions 
of  Lipschitz  continuity  of  /,L,  and  /i,  and  non-Zenoness  and  transversality  with 
17/  of  e-optimal  trajectories,  that  a  particular  discrete  approximation  V  of  the 
value  function  converges  to  the  viscosity  solution  of  HJB. 

3  From  Hybrid  Automata  to  Finite  Automata 

In  [2]  we  proposed  a  mapping  from  the  continuous  optimal  control  problem  (2)- 
(4)  to  a  hybrid  optimal  control  problem.  The  first  step  is  to  restrict  the  class  of 
controls  over  which  the  cost  function  is  minimized  to  piecewise  constant  controls 
taking  values  in  a  set  C  17.  C  17  is  a  finite  approximation  of  U  having  a 
mesh  size  5  :=  sup^^jy  min^ex's  ||^“<^||-  Next  we  restrict  the  continuous  behavior 
to  the  set  of  vector  fields  {/(a:,  If  we  associate  each  vector  field  to  a 

location  of  a  hybrid  automaton  and,  additionally,  define  a  location  reserved  for 
when  the  target  is  reached,  we  obtain  a  hybrid  automaton 

H  —  {Ex  IR^,  Z^,  D,  Eh,  G,  R) 

which  has  the  following  components: 

State  set  Z  x  IR^  is  a  finite  set  Z  =  Z5  U  {a/}  of  control  locations  and  n 
continuous  variables  x  G  IR^.  (7/  is  a  terminal  location  when  the  continuous 
dynamics  are  stopped  (in  the  same  sense  that  the  dynamics  are  stopped  in 
the  continuous  optimal  control  problem). 

Events  Us  is  a  finite  set  of  control  event  labels. 

Vector  fields  D  :  E  A'(]R’^)  is  a  function  assigning  an  autonomous  vector 
field  to  each  location;  namely  D{a)  =  f{x,  a). 

Control  switches  Eh  C  Z  x  Z  is  a  set  of  control  switches,  e  =  (<j,  a')  is  a 
directed  edge  between  a  source  location  a  and  a  target  location  a'.  If  Eh((j) 
denotes  the  set  of  edges  that  can  be  enabled  at  a  G  Z,  then  Z/i(a)  := 
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{(cr,  cr')  I  cr'  G  i7  \  cr}  for  <j  G  ^^5  and  Eh,{(Tf)  =  0.  Thus,  from  a  source 
location  not  equal  to  a f,  there  is  an  edge  to  every  other  location  (but  not 
itself),  while  location  <j/  has  no  outgoing  edges. 

Enabling  conditions  G  :  Eh  ^  {ge}eeEh  is  a  function  assigning  to  each  edge 
e  an  enabling  (or  guard)  condition  Qe  C  IR^. 


The  enabling  conditions  are  unknown  and  must  be  synthesized  algorithmi¬ 
cally.  (See  [2]  for  how  the  enabling  conditions  are  extracted  once  the  discrete 
problem  is  solved.)  Trajectories  of  H  evolve  in  cr-steps  and  t-steps.  cr-steps  oc¬ 
cur  when  H  changes  locations  (and  the  control  changes  value,  since  there  are 
no  self-loops)  and  t-steps  occur  when  the  continuous  state  evolves  according  to 
the  dynamics  of  a  location  as  time  passes.  The  reader  is  referred  to  [2]  for  pre¬ 
cise  statements.  A  hybrid  trajectory  is  non-Zeno  if  between  every  two  non-zero 
duration  t-steps  there  are  a  finite  number  of  a-steps  and  zero  duration  t-steps. 

Let  A  represent  an  arbitrary  time  interval.  A  bisimulation  of  H  is  an  equiva¬ 
lence  relation  {EsX IR^)  x  (I^^  x IR"^)  such  that  for  all  states  pi,p2  6  E5X IR^, 
if  Pi  ~  p2  and  a  e  EsU  {A},  then  if  pi  A  p[,  there  exists  p'^  such  that  p2  A  pg 
and  p'l  ~  P2- 

One  sees  that  ~  encodes  cr-steps  and  t-steps  of  in  a  time  abstract  form 
by  partitioning  Es  x  IR^.  If  ~  has  a  finite  number  of  equivalence  classes,  then 
they  form  the  states  of  a  finite  automaton  A.  If  q  :=  [((J,a:)]  and  q'  :=  [(o-',a:')] 
are  two  different  equivalence  classes  of  then  A  has  an  edge  q  q'  if  there 
exists  {cT,y)  G  q  and  (cr',p')  G  q'  such  that  (a,  p)  — )•  is  a  a-step  or  t-step 

of  H,  We  define  the  set  of  interesting  equivalence  classes  of  denoted  Q,  as 
those  that  intersect  Eg  x  c/(i7),  and  we  identify  a  distinguished  point  (<J,  $)  G  q 
for  each  q  E  Q,  denoted  q  =  [(cr,  ^J. 

Consider  the  class  of  non-deterministic  automata  with  cost  structure  repre¬ 
sented  by  the  tuple 


A  =  (Q,Es,E,obs,Qf,L,h). 


Q  is  the  state  set  just  defined,  and  Es  is  the  set  of  control  labels  as  before. 
obs  :  E  Es  is  a  map  that  assigns  a  control  label  to  each  edge  and  is  given  by 
obs{e)  =  a',  where  e  =  iq,q'),  q  =  [(o-,^]  and  q'  =  [(f7',<^')].  Qf  is  an  over  (or 
under)  approximation  oi  Qf  =  {q  ^  Q  \  3x  e  Hf  .  {a,x)  e  q  }.  E  C  Q  xQ  is 
the  transition  relation  of  A  and  is  defined  assuming  that  each  enabling  condition 
is  initially  the  entire  region  J7.  The  identity  map  is  implemented  in  A  by  an 
over- approximation  in  terms  of  equivalence  classes  of  That  is,  for  a  ^  cr', 
([<^5^)1?  [(<^^^01)  ^  E  if  the  projections  to  IR"^  of  [cr^x)]  and  [((j',rr')]  have  non¬ 
empty  intersection.  This  over-approximation  introduces  non-determinacy  in  A. 
Let 


sup  {t  \  y  =  (t>t{x,a)  }, 

{(T,x),{a,y)eq 
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Let  e  =  {q^q')  with  q  =  [(^r, f)]  and  q'  =  [(c^^^01•  L  :  E  ^  JR  is  the  discrete 
instantaneous  cost  given  by 


L(e) 


(  TgL{^,  a)  if  (T  =  a' 
[  0  if  cr  /  (j'. 


(6) 


h  :  Q  — >•  ]R  is  the  discrete  terminal  cost  given  by 


hiq)  := 


A  transition  or  step  of  A  from  q  £  Q  to  q'  G  Q  with  observation  <j'  €  Es  is 

denoted  q  ^  q' .  li  cr  ^  cr'  the  transition  is  referred  to  as  a  control  switch,  and 
it  is  forced,  a  =  a'  the  transition  is  referred  to  as  a  time  step.  If  E{q)  is  the 
set  of  edges  that  can  be  enabled  from  q  ^  Q,  then  for  a  £  Es,  Ea{q)  =  {e  G 
E{q)  I  obs{e)  =  a}.  U  \Ea{q)\  >  1,  then  we  say  that  e  G  Ea{q)  is  unobservable  in 
the  sense  that  when  control  event  a  is  issued,  it  is  unknown  which  edge  among 
Ea{q)  is  taken.  (Note  that  unobservability  of  edges  refers  strictly  to  the  discrete 
automaton  A,  whereas  in  H  one  may  be  able  to  reconstruct  which  edge  was 
taken  using  continuous  state  information).  If  cr  =  a',  then  \E(y{q)\  =  1,  by  the 
uniqueness  of  solutions  of  ODE’s  and  by  the  definition  of  bisimulation. 

A  control  policy  c  :  Q  Es  is  a  map  assigning  a  control  event  to  each  state; 
c(q)  =  a  is  the  control  event  issued  when  the  state  is  at  q.  A  trajectory  tt  of  A 
over  c  is  a  sequence  tt  =  ^  ^  ^  •  •  • ,  C  Q.  Let  Ilciq)  be  the  set  of 

trajectories  starting  at  q  and  applying  control  policy  c,  and  let  TIc{q)  be  the  set 
of  trajectories  starting  at  q,  applying  control  policy  c,  and  eventually  reaching 
Q/.  If  for  every  g  G  Q,  tt  G  nd^q)  is  non-Zeno  then  we  say  c  is  an  admissible 
control  policy.  The  set  of  all  admissible  control  policies  for  A  is  denoted  C. 

A  control  policy  c  is  said  to  have  a  loop  if  A  has  a  trajectory  qo 

. .  q^  =  qo,  qi  £  Q.  A  control  policy  has  a  Zeno  loop  if  it  has  a  loop 

made  up  of  control  switches  and/or  zero  duration  time  steps  (i.e.  Tg  =  0)  only. 

Lemma  1.  A  control  policy  c  for  non- deterministic  automaton  A  is  admissible 
if  and  only  if  it  has  no  Zeno  loops. 


Proof  First  we  show  that  a  non-deterministic  automaton  with  non-Zeno  tra¬ 
jectories  has  a  control  policy  without  Zeno  loops.  For  suppose  not.  Then  a  tra¬ 
jectory  starting  on  a  state  belonging  to  the  loop  can  take  infinitely  many  steps 
around  the  loop  before  taking  a  non-zero  duration  time  step.  This  trajectory  is 
not  non-Zeno,  a  contradiction.  Second,  we  show  that  a  control  policy  without 
Zeno  loops  implies  non-Zeno  trajectories.  Suppose  not.  Consider  a  Zeno  trajec¬ 
tory  that  takes  an  infinite  number  of  control  switches  and/or  zero  duration  time 
steps  between  two  non-zero  duration  time  steps.  Because  there  are  a  finite  num¬ 
ber  of  states  in  Q,  by  the  Axiom  of  Choice,  one  of  the  states  must  be  repeated  in 
the  sequence  of  states  visited  during  the  control  switches  and/or  zero  duration 
time  steps.  This  implies  the  existence  of  a  loop  in  the  control  policy.  Either  each 
step  of  the  loop  is  a  control  switch,  implying  a  Zeno  loop;  or  the  loop  has  one 


180  M.  Broucke  et  al. 


or  more  zero  duration  time  steps.  But  the  bisimulation  partition  permits  zero 
duration  time  steps  only  if  =0,  which  implies  a  Zeno  loop.  □ 


Example  1.  Consider  the  automaton  in  Figure  1.  If  we  are  at  qi  and  the  control 
a' a' a  is  issued,  then  three  possible  trajectories  are  qi  ^  qs  ^  q4  q2,  Qi  ^ 

Q4  Qb  ^  (}2,  or  qi  qs  The  first  trajectory  has  a  zero  duration 

time  step.  The  control  is  inadmissible  since  the  last  trajectory  has  a  Zeno  loop. 

4  Dynamic  Programming 

In  this  section  we  formulate  the  dynamic  programming  problem  on  A.  This 
involves  defining  a  cost-to-go  function  and  a  value  function  that  minimizes  it 
over  control  policies  suitable  for  non-deterministic  automata. 

Let  TT  ~  qo  ^  qi . ..  gjv-i  ^  qN^  where  qi  =  [(o-*,  ^i)]  and  tt  takes  the  sequence 
of  edges  eie^  . .  We  define  a  discrete  cost-to-go  J  :  Q  x  C  — >  IR  by 

=  I  if  n,(q)  =  i7e(9) 

[  oo  otherwise 

where  Nt^  =  min{j  >  0  |  G  Q/}-  We  take  the  maximum  over  Hdq)  because  of 
the  non-determinacy  of  A:  it  is  uncertain  which  among  the  (multiple)  trajectories 
allowed  by  c  will  be  taken  so  we  must  assume  the  worst-case  situation.  The 
discrete  value  function  V  :  Q  IR  is 

V{q)  =  min  J{q,c) 

cGC 

fox  q  e  Q\Qf  and  V{q)  =  h{q)  fox  q  G  Qf.  We  showed  in  [2]  that  V  satisfies  a 
DP P  that  takes  into  account  the  non-determinacy  of  A  and  ensures  that  optimal 
control  policies  are  admissible.  Let  Aq  be  the  set  of  control  assignments  c{q)  G  Es 
at  q  such  that  c  is  admissible. 
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Proposition  1.  V  satisfies 


V(q)  =  min 

c{q)eA, 

1  max  {B{e) +  V{q')}Y  q^Q\Qf 

le=(q,q')eE^,iq)^  J 

(7) 

II 

qeQf. 

(8) 

5  Non-deterministic  Dijkstra  Algorithm 


The  dynamic  programming  solution  (7)-(8)  can  be  viewed  as  a  shortest  path 
problem  on  a  non-deterministic  graph  subject  to  all  optimal  paths  satisfying  a 
non-Zeno  condition.  We  propose  an  algorithm  which  is  a  modification  of  the 
Dijkstra  algorithm  for  deterministic  graphs  [6].  First  we  define  notation.  Fn  is 
the  set  of  states  that  have  been  assigned  a  control  and  are  deemed  “finished” 
at  iteration  n,  while  Un  are  the  unfinished  states.  At  each  n,  Q  =  [7^  U  Fn- 
Q  is  the  set  of  control  events  at  iteration  n  that  take  state  q  to  finished 
states  exclusively.  Un  is  the  set  of  states  for  which  there  exists  a  control  event 
that  can  take  them  to  finished  states  exclusively.  Vn{q)  is  a  tentative  cost-to-go 
value  at  iteration  n.  Bn  is  the  set  of  “best”  states  among  Un. 

The  non-deterministic  Dijkstra  (NDD)  algorithm  first  determines  Un  hy 
checking  if  any  q  in  Un  can  take  a  step  to  states  belonging  exclusively  to  Fn- 
For  states  belonging  to  Un,  an  estimate  of  the  value  function  V  following  the 
prescription  of  (7)  is  obtained:  among  the  set  of  control  events  constituting  a 
step  into  states  in  Fn,  select  the  event  with  the  lowest  worst-case  cost.  Next,  the 
algorithm  determines  Bn,  the  states  with  the  lowest  V  among  Un,  and  these  are 
added  to  Fn+i.  The  iteration  counter  is  incremented  uritil  it  reaches  N  =  |Q|. 
It  is  assumed  in  the  following  description  that  initially  U(g)  =  oo  and  c{q)  =0 
for  all  q  E  Q. 

We  prove  that  algorithm  NDD  is  optimal]  that  is,  it  synthesizes  a  control 
policy  so  that  each  q  €  Q  reaches  Qf  with  the  best  worst-case  cost.  We  observe 
a  few  properties  of  the  algorithm.  First,  if  all  states  of  Q  can  reach  Qf  then 
Q  —  Qf  =  UnBn-  Second,  as  in  the  deterministic  case,  the  algorithm  computes 
V  in  order  of  level  sets  of  V.  In  particular,  V(Bn)  ^  V{Bn+i).  Finally,  we  need 
the  following  property. 

Lemma  2.  For  all  q  ^  Q  and  a'  G  Es, 

V{q)<  max  {L(e)  +  ^(9')}- 

e={g,q')€E^,{q) 


Proof.  Fix  q  €  Q  and  a'  e  Es.  There  are  two  cases. 
Case  1. 


V{q)<  mM  {V(9')}- 

e={q,q  )^E^>{q) 


In  this  case  the  result  is  obvious. 
Case  2. 


V{q)> 


max 

e=^{q,q')eE^>(qy 


(9) 
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Procedure  NDD: 

Fi  =  Qf;  Ui  =  Q  ~  Qf; 
for  each  q  6  Qf,  V(q)  =  h{q); 

for  n  =  1  to  N,  do 
for  each  q  G  Un, 

X'n(q)  =  {ct'  G  I  if  q  q',then  q'  G  Fn}; 

On  =  {qGUnJ  i:n(q)#0}: 
for  each  q  G  On, 

V„(q)  =  min<,-sr„(,,{maXe=(,,,.)£E„,(q){L(e)  +  V(q')}}; 
B„  =  argmin,go„{V„(q)}; 
for  each  q  G  Bn , 

N/Cq)  =  V„(q); 

c(q)  =  argmin,,6E„,<,){max„,,,,,)gE^,(,,{L(e)  +  V(q')}}; 

end  for 

Pn+l  —  Fn  U  Bnl  Un+1  Q  —  Fp  +  li 
endfor 


We  observed  above  that  q  belongs  to  some  Bn-  Suppose  w.Lo.g.  that  q  G  Bj. 

Together  with  (9)  this  implies  q'  G  Fj  for  all  q'  such  that  q  ^  q'.  This,  in  turn, 
means  that  cr'  G  Fj{q)  and  according  to  the  algorithm 

V{q)==Vj{q)<  max  {L(e)  +  i>{g')} 

e={q,q')^E^'(Q) 

which  proves  the  result.  □ 

Theorem  1.  Algorithm  NDD  is  optimal  and  synthesizes  a  control  policy  with 
no  Zeno  loops. 

Proof.  First  we  prove_optimality.  Let  V{q)  be  the  optimal  (best  worst-case)  cost- 
to-go  for  G  Q  and  Q  =  {q  ^  Q  |  V(q')  <  V{q)}.  Let  /(TTg)  be  the  number  of 
edges  taken  by  the  shortest  optimal  (best  worst-case)  trajectory  iTg  from  q.  Define 
q  =  argmin^^g{/(7rg)}.  Suppose  that  the  best  worst-case  trajectory  starting  at 

qis  'Kq  =  q^q-^ - We  showed  in  the  previous  lemma  that 

V{q)<  {L{€)  +  V{q')]  <L{e)  +  V{^). 

e={q,q')eE^,{q) 

Since  tt^  is  the  best  worst-case  trajectory  from  q  and  by  the  optimality  of  V{q) 
V{q)=  {L{e)  +  V(q')}  =  L(e)  +  V{^). 

e=(q,q')GE^,{q) 

Since  tt^  is  the  shortest  best  worst-case  trajectory,  we  know  that  ^  ^  Q,  so 
V{q)  =  V{q).  This  implies  V {q)  <  L{e)  -\-V{q)  =  V{q)^  a  contradiction. 
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To  prove  that  the  algorithm  synthesizes  a  policy  with  no  Zeno  loops  we  argue 
by  induction.  The  claim  is  obviously  true  for  Fi.  Suppose  that  the  states  of 
have  been  assigned  controls  forming  no  Zeno  loops.  Consider  Each  state 

of  Bn  takes  either  a  time  step  or  a  control  switch  to  Fn  so  there  cannot  be  a 
Zeno  loop  in  Bn-  The  only  possibility  is  for  some  q  e  Bn  to  close  a  Zeno  loop 
with  states  in  Fn-  This  implies  there  exists  a  control  assignment  that  allows  an 
edge  from  Fn  to  q  to  be  taken;  but  this  is  not  allowed  by  NDD,  Thus,  Fn+i  has 
no  Zeno  loops.  □ 

Remarks: 

1.  It  is  intuitively  reasonable  that  the  algorithm  cannot  synthesize  a  controller 
with  Zeno  loops.  This  worst-case  behavior  would  show  up  in  the  value  func¬ 
tion,  forcing  it  to  be  infinite  for  states  that  can  reach  the  loop. 

2.  When  we  say  that  the  algorithm  is  optimal,  we  mean  the  algorithm  de¬ 
termines  the  best  worst-case  cost  to  take  each  state  to  the  target  set.  In 
fact,  (see  remark  below)  the  hybrid  system  or  continuous  system  using  the 
synthesized  controller  may  perform  better  than  worst  case. 

3.  The  non-deterministic  automaton  predicts  more  trajectories  than  what  ei¬ 
ther  the  continuous  system  or  the  hybrid  system  can  exhibit.  Indeed,  the 
automaton  may  exhibit  a  trajectory  that  reaches  the  target  set  using  only 
control  switches,  and  thus  accruing  zero  cost.  This  is  not  of  concern.  Such 
a  trajectory  is  an  artifact  of  the  non-determinacy  of  the  automaton,  and  is 
not  used  in  the  determination  of  the  value  function,  which  accounts  only  for 
worst-case  behavior,  nor  is  it  exhibited  in  either  the  hybrid  system  or  the 
continuous  system  when  the  control  policy  synthesized  by  Algorithm  NDD 
is  used. 

4.  Related  to  the  previous  remark  is  that  the  non-deterministic  automaton  may 
also  predict  worst-case  behavior  which  is  not  exhibited  by  the  continuous 
system.  It  would  appear  that  a  discrepancy  will  develop  between  the  cost-to- 
go  obtained  by  applying  the  synthesized  controller  to  the  continuous  system 
and  the  cost-to-go  predicted  by  the  nondeterministic  automaton.  This  error 
is  incurred  every  time  a  control  switch  is  taken  and  is  effectively  an  error 
in  predicting  the  state  and  has  an  upper  bound  of  (5  at  each  iteration.  This 
error  was  accounted  for  in  our  proof  of  convergence  of  the  method,  and  the 
convergence  result  essentially  depends  on  the  fact  that  only  a  finite  number 
of  control  switches  occur  [2] . 

6  Example 

We  apply  our  method  to  the  time  optimal  control  problem  of  a  double  integrator 


Xi  =  X2 

±2  =  u. 

Given  the  set  of  admissible  controls  U  =  {u  :  |u|<l},  we  select  Q  = 

(—1,1)  X  (—1,1)  and  17/  =  ^€(0),  the  closed  epsilon  ball  centered  at  0.  The 
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cost-to-go  function  is  J{x,fi)  =  dt.  The  bang-bang  solution  obtained  us¬ 

ing  Pontryagin’s  maximum  principle  is  well  known  to  involve  a  single  switching 
curve.  The  continuous  value  function  V  is  shown  in  Figure  2(a). 


Fig.  2.  Continuous  and  discrete  value  functions  for  double  integrator 


Fig.  3-  Hybrid  automaton  for  time  optimal  control  of  a  double  integrator  system 


To  construct  the  hybrid  automaton  H  we  select  Es  =  {—1,1}.  H  is  show 
in  Figure  3.  The  state  space  is  {a^i  =  -l,c7i  :=  l,cr/}  x  IR^.  and  are 
unknown  and  must  be  synthesized,  while  =  ge^  —  Of  • 
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A  first  integral  for  vector  field  xi  =  0:2,  ^2  =  1  is  —  ~X2  =  ci,  ci  C  IR.  For 
±1  =  X2,  X2  =  —1  a  first  integral  is  xi  +  =  C2,  C2  G  IR.  We  select  a  transverse 

foliation  (see  [1])  for  each  vector  field,  given  by  X2  =  C3. 

We  define  Q,  Q/,  E,  L  and  h  for  automaton  A  derived  from  H  in  Figure  3. 
Q  can  be  visualized  using  Figure  4. 

The  states  g  €  Q  are  of  the  form  (cr,  [a:])  with  a  e  For  the  case 

cr  =  (ji  with  Cl,  C2  G  IR,  [x]  is  either  an  open  subset  of  IR^  bounded  by  the  leaves 
Cl  <  xi~hx2  <  Cl  +  A  and  C2  <  X2  <  C2  A  A;  01  an  open  interval  in  a  horizontal 
leaf  xi  —  ^X2  =  Cl,  C2  <  X2  <  C2  +  or  an  open  interval  in  a  vertical  leaf 
Cl  <xi  -  \x2  <  Cl  A  A,  X2  =  C2;  or  a  point  xi  ~  =  ci,  X2  =  C2.  Analogous 

expressions  can  be  written  for  a  =  cr_i.  In  Figure  4,  A  =  0.25,  ci  €  [—1,1] 
and  C2  G  [-1,1]-  If  we  identify  equivalence  classes  (c,  [x])  by  their  Euclidean 
coordinates  (ci,C2)  directly,  then  Q/,  shown  in  Figure  4  as  the  regions  inside  the 
dotted  lines,  includes  states  (cr,  [a:]),  where  [x]  satisfies  ci,C2  G  {—A,  A). 


Fig.  4.  Partitions  for  states  ai  and  a-i  of  the  hybrid  automaton  of  Figure  3 


Let  us  consider  the  edges  corresponding  to  control  switches  of  A.  q  = 
(ci,  [x])  €  Q  has  an  outgoing  edge  to  q'  =  (cr_i,  [^])  e  Q  if  [a:]  H  [?/]  ^0.  For 
example,  for  q  =  (cti,  [x])  and  [a:]  satisfying  ci  G  (—.25,  —.5)  and  C2  =  .25,  there 
are  three  outgoing  edges  from  q  to  q'^^i  =  1,. . ,  ,3,  with  fy]  satisfying  C2  =  .25 
and  Cl  €  (-.5, —.25),  ci  =  -.25,  and  ci  G  (-.25,0),  respectively.  Similarly,  for 
q  =  (^i,N)  H  satisfying  Ci  G  (—.5, —.25)  and  C2  G  (.75,1),  there  are  five 
outgoing  edges  from  q  to  q[,i  =  1,...  ,5,  with  [y]  satisfying  C2  G  (.75,1)  and 
Cl  G  (-.25,0),  Cl  =  0,  Cl  €  (0,  .25),  Ci  =  .25  and  ci  G  (-25,  .5),  respectively. 
Edges  corresponding  to  time  steps  of  A  can  be  determined  from  visual  inspec¬ 
tion  of  Figure  4.  For  example,  for  q  =  (cri,  [x])  with  [x]  satisfying  ci  G  (—.25,  —.5) 
and  C2  =  .25,  there  is  an  outgoing  edge  from  q  to  q'  =  (cri,  [y])  with  [y]  satisfying 
Cl  G  (-.25,  -.5)  and  C2  G  (.25,  .5). 

The  results  of  algorithm  NDD  are  shown  in  Figure  2(b)  and  Figure  5.  In 
Figure  5  the  dashed  line  is  the  smooth  switching  curve  for  the  continuous  prob¬ 
lem.  The  black  dots  identify  equivalence  classes  where  NDD  assigns  a  control 
switch.  Considering  ^e_i  we  see  that  the  boundary  of  the  enabling  condition  in 
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the  upper  left  corner  is  a  jagged  approximation  using  equivalence  classes  of  the 
smooth  switching  curve.  Initial  conditions  in  the  upper  left  corner  just  inside 
the  enabling  condition  must  switch  to  a  control  of  u  =  -1,  otherwise  the  trajec¬ 
tory  will  increase  in  the  X2  direction  and  not  reach  the  target.  Initial  conditions 
in  the  upper  left  corner  just  outside  the  enabling  condition  must  allow  time  to 
pass  until  they  reach  the  enabling  condition,  for  if  they  switched  to  u  =  -1 
they  would  be  unable  to  reach  the  target.  Hence  the  upper  left  boundary  of  the 
enabling  condition  is  crisp.  The  lower  right  side  of  the  enabling  condition  which 
has  islands  of  time  steps  shows  the  effect  of  the  non-determinacy  of  automaton 
A.  These  additional  time  steps  occur  because  it  can  be  less  expensive  to  take  a 
time  step  than  to  incur  the  cost  of  the  worst  case  control  switch.  Indeed  consider 
an  initial  condition  in  Figure  5(a)  which  lies  in  an  equivalence  class  that  takes  a 
time  step  but  should  take  a  control  switch  according  to  the  continuous  optimal 
control.  Such  a  point  will  move  up  and  to  the  left  before  it  takes  a  control  switch. 
By  moving  slightly  closer  to  the  target,  the  worst-case  cost-to-go  incurred  in  a 
control  switch  is  reduced.  Notice  that  all  such  initial  conditions  eventually  take 
a  control  switch.  This  phenomenon  of  extra  time  steps  is  a  function  of  the  mesh 
size  S:  3s  S  decreases  there  are  fewer  extra  time  steps.  Finally  we  note  that 
the  two  enabling  conditions  have  an  empty  intersection,  as  expected  in  order  to 
ensure  non-Zeno  trajectories. 


(a)  Pe_i 


(b)  Qei 


Fig.  5.  Enabling  conditions 


Figure  6  shows  trajectories  of  the  closed-loop  system  using  the  controller 
synthesized  by  NDD.  The  bold  lines  are  the  trajectories,  the  central  hatched 
region  is  an  enlarged  target  region,  and  the  shaded  areas  are  the  equivalence 
classes  visited  during  the  simulation. 
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Fig.  6.  Trajectories  of  the  closed- loop  system 


7  Conclusion 

In  this  paper  we  developed  an  efficient  single- pass  algorithm  to  solve  a  dynamic 
programming  problem  on  a  non-deterministic  graph  that  arises  in  the  solution  of 
a  continuous  optimal  control  problem  using  hybrid  systems  and  bisimulation.  We 
have  seen  that  the  single-pass  nature  of  the  solution  depends  on  the  partitioning 
method.  An  area  for  future  investigation  is  exploring  other  partition  methods  in 
relation  to  the  efficiency  of  the  algorithmic  solution  of  the  dynamic  programming 
problem.  This  would  include  partitions  that  are  not  bisimulations,  especially 
when  analytical  expressions  for  first  integrals  are  difficult  to  obtain. 

We  have  developed  a  prototype  tool  for  the  synthesis  of  hybrid  optimal  con¬ 
trols  based  on  bisimulation.  The  algorithm  has  complexity  0{N  log  N)  where  N 
is  the  number  of  states  of  the  automaton.  The  number  of  states  is  exponential 
in  the  dimension  of  the  continuous  state  space.  In  the  “vanilla”  version  of  our 
approach,  the  automaton  is  constructed  before  running  the  Djikstra-like  algo¬ 
rithm.  To  improve  the  speed  and  the  memory  usage  of  the  algorithm,  we  plan  to 
build  the  automaton  on  the  fly  while  algorithm  NDD  is  executing.  In  addition, 
we  plan  to  apply  the  approach  to  solving  a  number  of  optimal  control  problems 
arising  in  automotive  engine  control. 
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Abstract.  In  this  paper  we  investigate  some  analysis  and  control  prob¬ 
lems  for  discrete-time  hybrid  systems  in  the  piece-wise  affine  form.  By 
using  arguments  from  the  dissipativity  theory  for  nonlinear  systems,  we 
show  that  Hoo  analysis  and  synthesis  problems  can  be  formulated  and 
solved  via  Linear  Matrix  Inequalities  by  taking  into  account  the  switching 
structure  of  the  considered  system.  In  this  paper  we  address  the  gener¬ 
alized  problem  of  controlling  hybrid  systems  whose  switching  structure 
does  not  depend  only  on  the  state  but  also  on  the  control  input. 


1  Introduction 

Piece- Wise  Affine  (PWA)  systems  have  been  receiving  increasing  attention 
by  the  control  community  because  they  provide  a  useful  modeling  framework 
for  hybrid  systems.  In  fact,  discrete-time  PWA  systems  are  equivalent  to 
interconnections  of  linear  systems  and  finite  automata  [17],  to  complementarity 
systems  [9]  and  also  hybrid  systems  in  the  Mixed  Logic  Dynamical  (MLD)  form 
[1].  In  particular,  the  MLD  form  is  capable  to  model  a  large  class  of  hybrid 
systems  including  linear  hybrid  dynamical  systems,  hybrid  automata,  some 
classes  of  discrete-event  systems,  and  systems  with  qualitative  inputs/outputs 
[1,3].  The  algorithm  to  obtain  the  discrete-time  PWA  representation  of  an 
MLD  system  and  vice-versa  is  reported  in  [3].  In  order  to  stress  the  importance 
of  PWA  systems  it  is  worth  recalling  that  in  [2]  the  explicit  form  of  Model 
Predictive  Control  (MPC)  for  linear  constrained  systems  was  derived  and, 
besides  providing  an  algorithm  for  its  computation,  it  was  shown  that  the 
closed-loop  system  has  a  PWA  structure.  Also  in  this  case  the  closed-loop 
system  turns  out  to  be  a  PWA  model. 

An  important  feature  of  a  PWA  model  is  that  the  state-update  map  can  be 
discontinuous  along  the  boundary  of  the  regions.  For  instance,  when  considering 
PWA  systems  stemming  from  hybrid  systems  in  the  MLD  form,  discontinuities 
can  arise  from  the  representation  of  logic  conditions. 

The  control  synthesis  problem  for  MLD  systems  and  consequently  PWA  systems 
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is  computationally  difficult:  in  [1]  a  Mixed  Integer  Quadratic  Programming 
(MIQP)  approach  is  proposed  in  order  to  solve  the  control  problem  of  MLD 
systems  by  means  of  MPC  techniques.  Needless  to  say,  the  computational 
complexity  of  this  approach  may  increase  exponentially  with  the  prediction 
horizon  considered.  The  use  of  Linear  Matrix  Inequalities  (LMI)  techniques,  for 
which  computationally  advantageous  and  numerically  reliable  algorithms  as  well 
as  toolboxes  are  available  (see  [8])  would  seem  to  be  a  promising  alternative. 
Concerning  the  stability  analysis  of  PWA  systems,  the  authors  presented  various 
algorithms  with  different  degrees  of  conservativeness  in  [15].  Similarly  to  [12, 
13],  where  a  particular  class  of  continuous-time  PWA  systems  was  considered, 
such  procedures  exploit  Piece-Wise  Quadratic  (PWQ)  Lyapunov  functions  that 
can  be  computed  as  the  solution  of  a  set  of  LMIs.  For  the  sake  of  completeness, 
the  main  stability  test  of  [15]  is  reported  in  Section  2  in  a  suitable  form. 

In  this  work,  we  consider  both  analysis  and  synthesis  problems  for  the  general 
class  of  PWA  models  whose  switching  sequence  depends  on  both  state  and  input 
trajectories.  As  pointed  out  in  [3]  the  dependence  of  the  switching  sequence 
on  the  input  can  be  met  by  translating  an  MLD  system  into  a  PWA  form. 
Moreover,  the  dependence  of  the  switching  sequence  on  the  input  signal  is 
common  in  real  systems:  for  example,  it  could  be  caused  by  saturation  effects 
or  limitations  on  the  control  signal. 

It  is  worthwhile  emphasizing  that  this  type  of  PWA  models  is  more  general 
than  that  considered  in  [12,13]  and  [15]:  indeed,  in  these  works  the  switching 
structure  depended  on  the  state  only.  Furthermore,  we  generalize  the  results  of 
[15]  by  considering  analysis  and  synthesis  problems  with  performance  for  PWA 
systems. 

We  focus  on  the  Hoo  norm  showing  that  the  Foo-analysis  and  the  iJoo -synthesis 
of  a  piecewise  linear  state- feed  back  can  be  addressed  by  resorting  to  LMI-based 
algorithms.  The  rationale  of  our  derivation  hinges  on  the  use  of  passivity  theory 
for  nonlinear  systems  [14].  We  point  out  that  a  significant  application  of  the 
iLoo  analysis  test  is  the  possibility  of  checking  a  posteriori  the  performance  of 
MPC  for  both  linear  and  MLD  systems.  As  mentioned  before  this  can  be  done 
by  exploiting  the  explicit  PWA  form  of  the  closed-loop  system. 

The  results  are  presented  in  Sections  3  and  4.  An  illustrative  example  is 
provided  in  Section  5. 


Notation:  The  symbol  *  will  be  used  in  some  matrix  expressions  to  in¬ 
duce  a  symmetric  structure.  For  example,  if  L  and  R  are  symmetric  matrices, 
then 


L  -j-  M  -j-  *  * 
N  R 

Moreover,  we  define 


L-f  M  + 

N  R 


(1) 


N 

A:=0 


(2) 
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2  Stability  and  State-Feedback  Stabilization  of  PWA 
Systems 


A  linear  discrete-time  piecewise  affine  system  is  defined  by  the  state-space  equa¬ 
tion 


1  —  -^i^k  "b  ^i'^k  ~b  for 


Xk 

Uk 


C  Xi 


(3) 


where  Xk  €  IR^  is  the  state  and  Uk  C  is  the  control  input.  The  set  X  C  IR’^"'"”^ 
of  every  possible  vector  [x'^  is  either  or  a  polyhedron  containing  the 

origin,  {Xi}f=i  is  a  polyhedral  partition  ^  of  X  and  6  IR"^,  are  constant  vectors. 
We  refer  to  each  Xi  as  a  cell.  Moreover,  in  order  to  simplify  the  exposition,  we 
assume  that  our  cells  are  polyhedra  defined  by  matrices  Ff ,  ,  /f  and  /“  as 

follows 

Xi  :=  { [x‘^  such  that  Ffx  >  /f  and  >  /“}  .  (4) 

The  results  presented  in  this  paper  can  be  extended  to  systems  whose  cells  Xi 
have  a  more  complicate  structure. 

Moreover,  it  is  worth  introducing  the  following  notation: 

Xi  :=  {x  such  that  F^x  >  f^}  (5) 


and 

Sj  :=  such  that  3a:, u  with  a:  G  Xj?  C  •  (6) 

In  a  nutshell,  Sj  is  the  set  of  all  indices  i  such  that  Xi  is  a  cell  containing  a 
vector  [a:^  for  which  the  condition  x  G  Xj  is  satisfied.  We  denote  with  X  = 

{1, . . .  ,  s}  the  set  of  indices  of  the  cells  Xi  whereas  the  symbol  J  =  {1, . . .  ,t} 
will  be  used  to  denote  the  set  of  indices  of  the  cells  Xj‘  R  is  important  to  observe 
that: 


U5,=I.  (7) 

Furthermore,  if  cells  Xi  have  the  structure  pointed  out  in  eq.  (4)  then  the  sets 
Sj  are  disjoint  whereas  if  cells  Xi  have  a  more  complicate  structure  (for  instance 
when  mixed  state-input  constraints  are  used  to  define  each  cell  Xi)  then  the  sets 
Sj  could  be  overlapping.  In  the  latter  case  the  results  we  are  going  to  present 
could  become  more  conservative. 

When  we  focus  on  the  stability  of  the  origin,  we  consider  autonomous 


^  Each  set  xi  is  a  (not  necessarily  closed)  convex  polyhedron  s.t.  Xi  flXi  =  0,  Vi  7^ 

ULiX.^x. 
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PWA  systems  and  we  assume  that  rr  =  0  is  an  equilibrium  point.  To  begin  with 
it  is  necessary  to  observe  that  an  autonomous  system  can  be  obtained  from 
system  (3)  by  applying  a  suitable  control  law.  In  the  following  we  consider  a 
piecewise  linear  state  feedback  with  the  structure 


Uk  =  KiXk,  for 


'^k 


^  Xi’ 


(8) 


By  applying  the  controller  (8)  to  the  system  (3)  we  achieve  the  following  closed- 
loop  dynamic  system 


Xk+i  =  AiXk  +  ai,  for 


Xk 

Uk 


^  Xi 


(9) 


where  Ai  —  Ai  +  BiKi  and  Uk  =  KiXk.  We  note  that  the  the  evolution  of 
closed-loop  system  (9)  depends  on  the  “hidden”  variable  Uk  since  it  influences 
the  index  i  of  the  current  cell  Xi- 

As  customary  for  constrained  systems,  we  assume  that  the  state  trajectories 
[xk  ul]  generated  by  the  control  law  (8)  satisfy  [x^  u^]'^  €  X,  Vk  e  JN, 


In  [15]  the  stability  of  the  origin  of  PWA  system  was  characterized  by 
using  Piece-Wise  Quadratic  (PWQ)  Lyapunov  functions.  In  the  following 
theorem  we  report  the  main  result  of  [15]  valid  for  the  case  a*  =  0,  Vi  G  X  and 
adapted  to  the  closed  loop  system  (9). 

Theorem  1,  Consider  the  system  (9).  If  there  exist  matrices  Pi  =  Pj  >  0, 
Vi  G  X  such  that  the  positive- definite  function  V {x,  u)  =  x'^Pix,  V  G  Xi 

satisfies  V{xk-\-i,Uk-\-i)  -V{xk,Uk)  <  0,  then  the  origin  of  the  PWA  systems  (9) 
is  exponentially  stable  and  lim^_).4.oo  ||a:fcl|  =  0  for  all  system  trajectories  fulfilling 
[xj  ulf  6  X,  Vfe  e  IN.  □ 

The  Lyapunov  function  appearing  in  Theorem  1  can  be  computed  by  solving  the 

LMIs 


AjPiAj-Pj<0,  \/ii,j)eS 
Pi=P^  >0,  Vi  el 

where 


(10) 

(11) 


:=  |(Li)  :  iJ 


G  X  and  3k  G  No,  3 


Xk 

Xk+1 

Uk^ 

J 

Uk+1_ 

such  that 


Xk 

Uk 


G  Xj  and 


Xk+l 

Uk+1 


^Xi 


(12) 


In  other  words,  the  set  S  contains  all  the  ordered  pairs  of  indices  denoting  the 
possible  switches  from  cell  j  to  cell  i  and  it  can  be  computed  via  reachability 
analysis  for  MLD  systems  [4],  Then,  the  inequalities  (10)  take  into  account  all  the 
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admissible  switches  between  different  regions  and  guarantee  that  the  Lyapunov 
function  is  decreasing  along  all  possible  state  trajectories.  When  there  exist 
matrices  Pi  such  that  the  LMIs  (10)  and  (11)  are  satisfied,  the  PWA  system 
and  the  corresponding  controller  (eq,  (8))  are  termed  PWQ-stable  and  PWQ- 
stabilizing  respectively.  We  refer  the  interested  reader  to  [15]  for  further  details. 


Remark  1.  Conservativeness. 

The  conservativeness  of  the  LMIs  conditions  for  stability  analysis  can  be  reduced 
by  exploiting  the  so-called  S-Procedure  [20]  in  order  to  avoid  imposing  Pix  >  0 
for  [x'^  G  Xj-)  3  ^  ^  [15]'  This  modification  was  proposed  in  [12]  for 

continuous-time  PWA  systems  and  can  be  easily  generalized  to  the  discrete¬ 
time  case.  We  point  out  that  similar  modifications  can  be  applied  to  all  the 
analysis  LMIs  we  derive  in  the  following. 

It  is  important  to  highlight  that  with  respect  to  the  continuous-time  case  (see 
[12])  in  the  discrete-time  case  there  is  no  need  to  guarantee  the  continuity  of 
the  Lyapunov  function  over  the  whole  state-space.  This  fact  can  determine  a 
reduced  degree  of  conservativeness  of  the  results  that  we  are  going  to  present 
with  respect  to  those  presented  in  [12]. 

Finally,  following  the  lead  given  in  [11],  the  authors  proposed  in  [7]  discrete-time 
performance  analysis  results  with  a  notably  reduced  degree  of  conservativeness. 

□ 


Remark  2.  Extension  of  Theorem  1. 

Theorem  1  can  be  extended  to  the  case  ^  0  as  done  in  [12,13]  by  introducing 
the  extended  state  Xk  =  [xJ.  1]^  and  rewriting  the  system  (3)  as  follows: 


“b  Ei'^k 


Xk 

Uk 


where 


(13) 


(14) 

□ 


When  designing  the  controller  i.  e.  when  the  controller  gains  Ki  appearing  in 
the  inequalities  (10)  are  unknown,  the  set  of  all  possible  switches  is  generally  not 
known  in  advance,  and  it  could  be  necessary  to  consider  all  the  pairs  of  indices 
in 

Call  :=  X  X  Z 


instead  of  S. 

Furthermore,  we  note  that  the  design  of  a  controller  of  type  (8)  could  be 
a  very  hard  task  because,  at  each  time  instant,  the  vector  has  to  be 
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calculated  by  means  of  a  control  gain  Kj  whose  index  i  is  found  on  the  basis  of 
the  admissibility  condition 


Uk 


€  Xi' 


(15) 


This  implies  that  in  general  it  is  not  possible  to  calculate  Uk  since  the  index  i  for 
which  the  condition  (15)  is  satisfied,  is  difficult  to  know  in  advance.  Therefore,  we 
turn  our  problem  into  one  of  designing  a  controller  with  the  following  structure 


Uk  =  KjXk,  Xkexj^ 


(16) 


Thus  we  consider  a  different  control  gain  not  for  all  the  cells  Xi  with  i  but 
for  all  cells  Xj  with  j  G  JT.  Despite  this  restricted  controller  structure,  in  order  to 
design  a  control  law  of  type  (16)  one  must  exploit  a  different  Lyapunov  matrix 
Pi  for  each  cell  Xi  with  i  G  X  (see  the  corresponding  analysis  result  of  Theorem 
1)  to  reduce  the  conservativeness. 


3  Synthesis  of  a  Stabilizing  State  Feedback 


In  this  section  we  consider  the  problem  of  finding  a  state  feedback  control  law 
of  type  (16)  for  the  system  (3).  For  this  purpose  we  start  from  the  analysis 
condition  (10)  rewritten  for  the  closed-loop  system: 


'^ij  X'k  ?  for 


Xk 

Uk 


^  Xi,xk  e  Xj 


(17) 


where  Aij  =  +  BiKj  and  Uk  =  KjXk^  More  precisely,  eq.  (10)  rewritten  for 

the  closed  loop  system  (17)  assumes  the  following  form 

~  Pi  <0  Vj  G  J,\fi  G  5j,V/  such  that  {l^i)  G  Sail,  (18) 
Pi  =  PT  >  0,  Vz  G  X.  (19) 

Inequalities  (18)-(19)  represent  a  closed-loop  stability  condition.  For  each  cell 
Xj  (with  j  G  ij)  we  want  to  calculate  a  state  feedback  control  law  represented 
by  the  gain  matrix  Kj.  The  control  gain  Kj  is  used  when  [x'^  belongs 
to  any  cell  Xi  such  that  i  G  Sj  or,  equivalently,  if  Xk  €  Xj-  Furthermore,  this 
controller  is  applied  independently  of  the  subcell  xi  in  which 
is  contained  (obviously,  the  pair  (/,  i)  has  to  belong  to  the  set  of  all  possible 
switches  i.e.  Sail).  Clearly,  in  view  of  eq.  (7)  these  inequalities  are  exhaustive 
stability  conditions  since  they  cover  all  possible  transitions  of  the  set  Sail. 


Because  each  matrix  Pi  is  positive  definite  we  can  rewrite  (18)  by  resort¬ 
ing  to  the  Schur  lemma  as  follows: 

<0,  Vj  G  X,Vz  G  such  that  (/, f)  G  cSa/f  (20) 


-Qi  QiAfj 

AijQi  Qi 
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where  Qi  :=  We  will  show  that  (20)  is  guaranteed  if  there  exist  matrices  Gj 
with  j  £  J  of  suitable  dimensions  such  that  the  following  alternative  inequalities 
are  satisfied 

]  e  J,Vi  G  Sj,Vl  such  that  (l,i)  G  Sail  (21) 

where  Gj,  j  €  J  are  matrices  of  suitable  dimensions.  In  order  to  demonstrate 
that  (21)  implies  inequalities  (20)  we  first  observe  that  matrices  Gj  are  nonsin¬ 
gular  since  we  have  assumed  Qi  >  0  Vi  C  X  whereas  the  element  {1, 1}  of  (21) 
implies  that  Gj+Gj  >  Qi.  Secondly,  if  Qj  >  0  the  matrix  {Gj -Qi)Qi^{Gj-Qi) 
is  nonnegative  definite  and  consequently; 

0<Gj+Gj  -Qi<GjQ-^Gj.  (22) 

Moreover,  because  of  (22)  inequalities  (21)  imply 

1  <  0.  such  that  {l,i)  G  (23) 

>A.ijG  j  Qi 

Finally,  recalling  that  the  matrices  Gj  are  nonsingular  we  can  obtain  (20)  from 
(23)  by  multiplying  (23)  from  the  right  by  diag  {QiGj'^j}  and  from  the  lea 

by  diag{Gj'^Qij}. 

These  considerations  lead  to  the  following  algorithm  to  calculate  a  stabilizing 
state-feedback  control  law.  Indeed,  in  the  following  theorem  we  propose  calculat¬ 
ing  a  state-feedback  controller  of  type  (16)  by  exploiting  a  Piece-Wise  Quadratic 
(PWQ)  Lyapunov  function  defined  by  s  matrices  Pi  with  i  gX: 

Theorem  2.  Consider  the  PWA  system  (3).  There  exists  a  state  feedback  con¬ 
trol  law  of  type  (16)  guaranteeing  PWQ  stability  if  there  exist  matrices  Qi  = 
Qj  >  0  with  i  gX  and  matrices  Gj ,  Yj  with  j  G  J,  such  that  Wj  G  jyi  G  Sj 
andMl  with  {l,i)  G  Sail 


(24) 

(25) 

□ 


4  Hoc  Performance  of  Piecewise  Affine  Systems 

Consider  the  PWA  system 


Xk+i  =  AiXk  +  BiUk  +  Bfwk  +  ai 
~  GiXk  “b  Di^k  ~b  Di  nJki 


Xk 

Uk 


e  Xu^k  e  Xj 


(26) 
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where  G  IR  is  a  disturbance  signal  and  zj^  €  IR^  is  a  performance  output 
that  can  model,  for  instance,  tracking  errors  or  the  cost  of  the  input.  First,  to 
simplify  the  exposition  we  consider  the  case  =  0,Vz  €  Z  (Subsection  4.1). 
Then,  we  extend  our  results  to  the  case  ai  ^  0  (Subsection  4.2).  In  any  case,  we 
assume  that  the  system  (26)  admits  j:  =  0  as  an  equilibrium  point. 

As  customary  in  control  of  nonlinear  systems  [14]  we  consider  performance  in¬ 
dices  defined  over  a  finite  time  horizon.  In  this  section  we  focus  on  the  distur¬ 
bance  attenuation  problem  in  an  framework:  given  a  real  number  7  >  0, 
the  exogenous  signal  w  is  attenuated  by  7  if,  assuming  3:0  =  O5  for  each  integer 
>  0  and  for  every  w  e  I2  ([0,  N] ,  IR’’) 

(27) 

k==0  k=0 


The  control  problem  of  discrete-time  nonlinear  systems  can  be  very  difficult  due 
to  the  lack  of  geometric  properties  [14].  We  will  show  that  for  PWA  systems 
this  task  turns  out  to  be  less  impervious  provided  the  use  of  some  fundamental 
LMI  techniques  [16,6]. 

To  begin  with,  we  present  some  analysis  results  for  the  following  closed- 
loop  system  obtained  by  applying  a  feedback  control  law  of  type  (16)  to  system 


Xk+i  =  AijXk  4-  Bfwk 
Zk  —  CijXk  +  D'^Wk, 


Xk 

Uk 


^Xi,^k  e  Xj 


(28) 


where  Aij  =  Ai-^ BiKj,  Cij  =  CiADiKj  and  Uk  =  KjXk^  We  observe  again  that 
the  evolution  of  the  closed-loop  system  (28)  depends  on  the  “hidden”  variable 
Uk  since  it  influences  the  index  i  of  the  cell  Xi- 

A  discrete-time  nonlinear  system  (as  the  PWA  system  (28))  is  strictly  dissipative 
with  supply  rate  W  :  IR®  x  IR”  — >■  IR  if  there  exists  a  non-negative  function 
V  :  IR’^  X  IR”^  —)« IR  termed  storage  function  such  that 


Ww  e  IR”,VA:  >  0,  V{xk+uUk+i)  -  V(xk,Uk)  <  W{zk,Wk)  (29) 

and  V(0,?x)  =  0,  Vu  [5].  Condition  (29),  is  the  so-called  dissipation  inequality 
that  can  be  equivalently  represented  through  the  condition  [14,19] 

N 

^Wk^'iN  >0yxo,  V{xN+uUN+i)  -V{xo,uo)  <J2^{zk,Wk).  (30) 

A;=0 

Hereafter  we  concentrate  on  finite  gain  dissipative  PWA  systems  with  the  fol¬ 
lowing  supply  rate 


^oo(z,w)  =  -  ||z||2),  7  >  0.  (31) 

As  will  be  shown,  the  supply  rate  W^{z,w)  is  related  to  the  performance 

of  the  PWA  system. 
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4.1  Hoo  Analysis  and  Synthesis  for  PWA  Systems  without 
Displacement  Terms 

The  rationale  presented  in  this  section  hinges  on  the  assumption  that  the  pair 
(Ai,  Bi),  ieXis  PWQ  stabilizable:  that  is,  we  assume  that  there  exists  Kj,  j  £j 
and  Pi  =  Pf  >  0  with  i  e  X  such  that  Vj  with  (/,  i)  C  Sail 

AlPiAij-Pi<0  (32) 

where  Aj  ==  +  BiKj  if  i  e  Sj  and  Sail  is  the  set  of  all  possible  switches. 

The  next  Lemma,  which  is  a  generalization  of  the  classical  Bounded  Real  Lemma 
[18,14]  to  PWA  systems,  allows  to  analyze  the  Hoo  performance. 

Lemma  1.  Consider  the  system  (28)  with  zero  initial  condition  Xq  =  0.  If  there 
exists  a  function  V{x^u)  —  x^PiX  for  C  Xi  '^Hh  Pi  —  PX  >  0  satisfy¬ 

ing  the  dissipativity  inequality  (29)  with  supply  rate  (31),  i.e. 

yk,V{xk^i,Uk+i)  -  V{xk,Uk)  <  {j'^Wwklf  -  ll^fcll^),  (33) 

then,  the  Hoo  performance  condition  (27)  is  satisfied. 

Furthermore,  condition  (33)  is  fulfilled  if  the  following  matrix  inequalities  are 
satisfied 

Vj  £j,Vie  Sj,\/l  with  (/,  i)  e  S,  Mi,ij  <  0.  (34) 


where 

Mi,ii +  BjPiAi  BfPiBi  +  DfDi  -  7=/ 
In  this  last  case  the  system  (28)  is  PWQ  stable. 

Proof  By  recalling  that  xq  =  0,  from  (33)  it  follows  that,  ViV  >  0 


(35) 


V(XK+I,«iv+1)  <  E  - 11^*11')  ■ 

A:=0 

Since  V^(a^iv-(-i5  ^iv+i)  is  well  defined  and  positive  it  follows  that  condition  (27) 
is  met.  ^  ^ 

Moreover,  if  we  assume  ^  Xi  ^md  [^'k+i  ^m-i]  ^  Xi  we  can  write 

inequality  (33)  as: 

Vuifc,  [xj  wj]  Mi^ij  [xl  wlf  <  0.  (37) 

Obviously,  inequality  (37)  is  satisfied  if  condition  (34)  is  met.  On  the  other  hand, 
it  holds  that  Cjfiij  >  0.  Consequently  by  considering  the  element  (1,1)  of  (34) 
we  can  state  that 

Vj  ej,'iie  Sj,'il  with  (l,i)  e  S,AlPiAij  -  Pi  <0. 


(38) 
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This  implies  that  the  system  (28)  is  PWQ  stable.  □ 

Next  we  focus  on  finding  a  state-feedback  control  law  of  the  type  (16)  for  the 
system  (26)  satisfying  a  suitable  requirement.  The  main  result  is  summarized 
in  the  following  theorem. 

Theorem  3.  Consider  the  PWA  system  (26).  There  exists  a  state  feedback 
control  law  of  type  (16)  guaranteeing  PWQ  Lyapunov  stability  and  fulfilling 
the  dissipativity  constraint  (29)  with  supply  rate  (31)  if  there  exist  matrices 
Qi  =  Qj  >0  with  i  e  I  and  matrices  Gj,Yj  with  j  e  J,  such  that 
'^3  e  J,Vi  G  Sj,yi  with  {l,i)  e  Sail 


Qi  -  Gj  -  G) 
AiGj  -h  BiVj 
CiGj  +  DiVj 


-Qi 

0 


* 

-I 


0  -7^/ J 


The  feedback  gains  Kj  with  j  £  J  are  given  by: 


(39) 


(40) 


□ 

The  proof  of  this  theorem  can  be  achieved  form  the  results  reported  in  Lemma 
1  by  applying  the  same  line  of  reasoning  used  to  demonstrate  Theorem  2. 


4.2  Extension  to  PWA  Systems  with  Displacement  Terms 

Some  analysis  results  have  been  extended  to  the  case  ^  0  by  considering 
an  extended  state  space  (see  eq.  (13))  [12,13].  Unfortunately,  this  approach  is 
very  restrictive  for  synthesis  problems  because  the  extended  dynamic  matrix 
Ai  is  never  a  stability  matrix  (Ai  contains  an  unreachable  eigenvalue  at  1)  and 
consequently  it  is  never  possible  to  find  P  =  >  0  satisfying  the  Lyapunov 

stability  condition 


AjPAi-P<0.  (41) 

On  the  other  hand,  the  set  Sail  of  all  possible  transitions  contains  also  the 
transitions  of  type  (z,i)  i.e.  from  region  i  to  the  same  region.  This  implies  that 
the  synthesis  approach  proposed  in  the  previous  part  of  this  section  can  never 
be  applied  to  a  system  obtained  by  extending  the  state  vector  as  proposed  in 
[12,13]. 

Therefore  we  consider  a  different  approach  based  on  the  extension  of  the  input 
signal  Wk  as  follows: 
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Thus,  the  system  (26)  can  be  rewritten  as; 


where 


Xk+i  =  AiXk  +  BiUk  +^B^Wk 
Zk  =  CiXk  +  DiUk  +  bfwk, 


Xk 

Uk 


G  Xu^k  G  Xj 


=  [Bf  /]  b^  =  [D^  0]  . 


(43) 


(44) 


The  Hoo  framework  considered  here,  is  based  on  a  finite  horizon  definition  of 
the  I2  gain  and,  consequently,  the  proposed  extension  of  the  disturbance  input 
is  sensible. 

Clearly,  it  is  possible  to  apply  the  control  approach  proposed  in  Theorem  3  di¬ 
rectly  to  the  extended  system  (43).  This  can  be  conservative  because  is  not  an 
unknown  disturbance  but  a  known  term.  Unfortunately,  in  general,  is  known 
only  when  the  control  signal  Uk  has  already  been  calculated.  Notwithstanding 
this,  under  the  standard  assumption 


tti  =  aj,Vz  €  Sj,Vj  C  ,  (45) 

an  alternative  control  strategy  can  be  proposed.  More  precisely,  the  control  is 
assumed  to  have  the  following  structure: 


u,^[K^K]] 


Xk  G  Xj- 


(46) 


In  this  way  the  controller  can  take  into  account  also  the  displacement  term 
ai  =  Dwk  where 

D:=  [0  /]. 

By  applying  the  control  law  (46)  to  the  PWA  system  (43)  we  obtain  the  closed- 
loop  PWA  system: 


where 


Xk+i  =  AijXk  AB'^jWk 
Zk  =CijXk+bfjWk, 


Xk 

Uk 


e  Xi,Xk  G  Xj 


Aij  =  Ai  +  BiKj  =  Br  +  BiKp 
Cij  =  Ci  +  DiK]  Vfj  =  bf  +  DiK^D. 


(47) 


(48) 


Now,  we  can  apply  the  Hoc  result  of  Lemma  1  to  the  closed-loop  PWA  system 
(47)  to  arrive  at  the  synthesis  procedure  summarized  in  the  subsequent  theorem. 
In  this  case,  the  controller  gain  is  composed  of  two  different  parts,  ATj  and  X?, 
that  constitute  two  unknowns  of  a  suitable  LMI  problem; 
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Theorem  4.  Consider  the  PWA  system  (26).  There  exists  a  state  feedback  con¬ 
trol  law  of  type  (46)  guaranteeing  PWQ  Lyapunov  stability  and  fulfilling  the 
dissipativity  constraint  (29)  with  supply  rate 


W^{zk,Wk)  =  (7^11  [wl  af  -  ||2if)  = 

=  (7^  (Ikfef  +  l|otf )  -  Ikfcf  ),7  >  0, 


Uk 


^Xi 


(49) 

(50) 


if  there  exist  matrices  =  Qf  >  0  with  i  e  1  and  matrices  Gj,Yj,K^  with 
j  e  J ,  such  that  Vj  e  J^i  e  Sj,\/l  with  {l,i)  e  Sail 


'Q,-Gj~Gj  * 

AiGj  +  BiYj  —Qi 

CiGj  +  DiYj  0 


* 

* 


-I 


<0. 


(51) 


[  0  {Bf  +  BiK]D^  ^  {br  +  DiK]D) ^  -'y‘^1 

The  feedback  gain  matrices  Kj  with  j  £  ff  are  given  by: 


K]:=^Y,G-\ 


(52) 


□ 


5  Numerical  Example:  The  Tank  Case 

The  example  we  consider  here  is  inspired  by  the  three-tank  benchmark  described 
in  [10]  that  will  be  the  subject  of  future  investigation.  It  consists  of  a  single  tank 
with  cross  section  section  A.  It  is  filled  by  means  of  a  pump  whose  mass  flow 


rate  is  given  by  the  control  variable  u  (see  Figure  1).  Obviously,  we  suppose  that 
0  <  w  <  Umax-  The  tank  level  is  denoted  by  x.  At  the  heights  Xi  and  X2  we 
assume  there  are  two  pipes  through  which  we  have  the  output  mass  flow  rates 
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Kix  and  K2X  respectively.  Finally,  at  the  bottom  we  have  a  constant  output 
mass  flow  rate  /.  In  our  case  we  have  chosen  the  following  numerical  values  with 
suitable  dimensions: 


A  =  l,  Ki=0.2,K2  =  0A, 

Xi  =  0.3,  X2  =  0.6,  /  =  0.01,  Umax  =  0.019. 


(53) 


In  order  to  introduce  the  tank  model  we  adopt  the  following  notation.  Let  b  be 
a  boolean  expression,  then  we  denote  with  |  •  |  the  function 


fl  if  b  =  TRUE 
1^1 -|0  if  b  =  FALSE 


(54) 


Then,  a  possible  continuous- time  model  for  the  tank  of  Figure  1  is  given  as 
follows: 


Ax  — uKlt  ^  0)  A  (w  ^  Umax')\  “1“  '^max\'^  ^  ”^7710x1  T 
-  /  -  Kix\x  >  Xi\  -  K2x\x  >  X2\. 

In  this  model  we  have  neglected  the  obvious  physical  condition  x  >  0.  Moreover, 
it  is  very  simple  to  obtain  from  (55)  the  following  PWA  continuous-time  model: 


-f 

if 

u  <  0, 

X  <  Xl 

U  -  f 

if 

0  <  ti  <  Umax, 

X  <  Xi 

Umax  f 

if 

U  >  Umax  > 

X  <  Xl 

-f~Kix 

if 

U  <  0, 

Xl  <  X  <  X2 

u  —  f  —  Kix 

if 

0  <  U  <  Umax, 

Xl  <  X  <  X2 

Umax  —  f  — 

if 

U  >  Umax  , 

Xl  <  X  <  X2 

-f^(Ki+K2)x 

if 

U  <  0, 

X  >  X2 

U~  f  -  {Kl  +  K2)x 

if 

0  ^  li  ^  Umax, 

X  >  X2 

Umax  —  f  — 

if 

U  ^  Umax  , 

X  >  X2. 

This  model  can  be  reduced  to  a  discrete-time  PWA  system  of  type  (26)  by 
discretization  (employing  the  implicit  Euler’s  rule  with  a  discretization  time 
equal  to  0.5  sec.).  Finally,  the  PWA  discrete-time  model  has  9  cells  Xi  ^ 
cells  Xj  '  Furthermore,  we  do  not  consider  any  disturbance  inputs  of  type  w  and 
we  consider  the  problem  of  regulating  the  level  z  :=  x  around  0.1.  For  this 


Fig.  2.  Closed  Loop  simulation  -  (a)  State,  (b)  Control  Input,  (c)  Switching  history 
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purpose  we  have  applied  to  the  discretized  model  an  Hoo  regulator  obtained  by 
means  of  the  synthesis  procedure  of  Theorems  3  and  4.  In  Figures  2.(a)-2.(b)  we 
report  the  time- histories  of  the  state  variable  and  of  the  control  input  (the  initial 
state  considered  is  xq  =  0.7).  Finally,  in  Figure  2.(c)  we  show  the  corresponding 
switching  history  (we  recall  that  we  have  9  cells  of  type  Xi  and  in  this  picture 
we  report  the  index  of  the  cell  Xi  in  which  the  vector  [x'^  is  contained). 


6  Conclusions 

In  this  paper  we  derived  LMIs-based  procedures  to  solve  Hqq  analysis  and  syn¬ 
thesis  problems  for  PWA  systems  whose  switching  sequence  depends  on  the  state 
and  on  the  control  input.  These  PWA  systems  can  be  found  by  translating  an 
MLD  system  into  PWA  form.  The  analysis  tests  can  be  applied  to  assess  the 
performance  of  MPC  control  schemes  applied  both  to  linear  and  hybrid  systems. 
Moreover,  the  state-feedback  design  methodologies  provide  an  alternative  way 
to  synthesize  controllers  with  a  prescribed  degree  of  performance.  All  the  pro¬ 
posed  synthesis  procedures  are  clearly  only  sufficient  i.e.  nothing  can  be  said 
if  the  LMIs  are  infeasible.  A  thorough  analysis  of  their  conservativeness  will  be 
subject  of  further  investigations. 
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Abstract.  It  has  been  observed  that  there  are  a  variety  of  situations 
in  which  the  most  popular  hybrid  simulation  methods  can  fail  to  prop¬ 
erly  detect  the  occurrence  of  discrete  events.  In  this  paper,  we  present 
a  method  for  detecting  discrete  which,  using  techniques  borrowed  from 
control  theory,  selects  integration  step  sizes  in  such  a  way  that  the  sim¬ 
ulation  slows  down  as  the  state  approaches  a  set  which  triggers  an  event 
(a  guard  set).  Our  method  guarantees  that  the  state  will  approach  the 
boundary  of  this  set  exponentially;  and  in  the  case  of  linear  or  polyno¬ 
mial  guard  descriptions,  terminating  on  it,  without  entering  it.  Given 
that  any  system  with  a  nonlinear  guard  description  can  be  transformed 
to  an  equivalent  system  with  a  linear  guard  description,  this  technique  is 
applicable  to  a  broad  class  of  systems.  Even  in  situations  where  nonlinear 
guards  have  not  been  transformed  to  the  canonical  form,  the  method  is 
still  increases  the  chances  of  detecting  and  event  in  practice.  We  show 
how  to  extend  the  method  to  guard  sets  which  are  constructed  from  many 
simple  sets  using  boolean  operators  {e.g.  polyhedral  or  semi-algebraic 
sets)  .  The  technique  is  easily  used  in  combination  with  existing  numer¬ 
ical  integration  methods  and  does  not  adversely  affect  the  underlying 
accuracy  or  stability  of  the  algorithms. 


1  Motivation  and  Previous  Work 

Numerical  simulation  is  an  important  tool  for  designing  and  analyzing  hybrid 
systems.  In  addition  to  simulation,  numerical  approximation  techniques  are  in¬ 
creasingly  being  used  in  approximate  reachability  computations,  verification  and 
other  forms  of  automated  analysis  [5],  [6],  [13j.  It  is  well  known  that  when  sim¬ 
ulating  hybrid  systems  failure  to  detect  an  event  can  have  disastrous  results  on 
the  global  solution  due  to  the  discontinuous  nature  of  the  problem.  Several  docu¬ 
ments  detailing  requirements  for  hybrid  simulators  list  accurate  event  detection 
as  one  primary  concern  [14],  [11]. 

Figure  la  illustrates  graphically  the  behavior  of  a  generic  hybrid  system 
model.  At  the  initial  time  to  5  the  mode  qi  is  active  and  the  continuous  system 
flows  according  to  the  differential  equation  x  =  fi{x)  with  initial  condition 
xq  =  x{to).  Once  the  condition  Guard  is  true  the  transition  from  qi  to  q2  is 
enabled;  the  state  may  be  reset  instantaneously  and  the  system  enters  mode  q2 
where  it  then  flows  according  to  x  =  f2{^)-  The  problem  we  are  concerned  with 
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a)  b) 


Fig.  1.  (a)  Conceptual  model  of  a  generic  Hybrid  System,  (b)  three  situations  for  which 
popular  simulators  fail  to  properly  detect  or  localize  events. 


is  correctly  detecting  the  discrete  transitions.  More  formally:  problem  Given 
f  \  BT-  xq  =  x{tQ)  e  and  g  :  R  such  that  ^(xo)  <  0,  simulate 

X  =  f{x),  for  the  time  interval  [to,t*]  where  t*  must  be  computed  as  the  first 
time  instant  such  that  g{x{t))  >  0.  problem  We  assume  the  guard  set  has  a 
non-empty  interior  and  is  described  as  Guard  =  {x  :  g{x)  >  0}  where  g(x)  is 
a  continuously  differentiable.  See  [12]  for  an  interesting  discussion  of  the  unique 
difficulties  associated  with  solving  such  problems.  It  is  well  known  that  systems 
of  differential  equations  with  nonlinear  guards  can  be  transformed  to  a  equivalent 
systems  with  linear  guards  by  appending  a  new  state  variable  2:  =  g{x)  then  the 
new  system  is 


X  =  f{x)  x  =  f{x),  z  =  ^  ■  fix) 

g(x)  >  0  z  >  0.  (1) 

Most  hybrid  system  simulators(  [1],  [9],  [18])  divide  the  task  into  an  event 
detection  phase  followed  by  an  event  localization  phase.  They  proceed  with  the 
detection  phase  by  checking  if  g{x{to))  >  0.  If  the  condition  is  false,  numerically 
integrate  the  differential  equation  through  one  time  step,  to  —  Iq  h  and 
check  if  g{x{ti))  >  0.  This  procedure  is  repeated  until  a  step  is  taken  for  which 
g(x{tk))  >  0  is  true,  at  which  point  an  event  is  assumed  to  have  occured  in  the 
interval  {tk-i,tk].  Note  that  the  step  size  h  is  selected  without  considering  the 
guard  dynamics.  Some  tools  then  activate  a  localization  phase  to  determine  the 
time  of  occurrence  more  precisely,  but  some  simply  assume  the  event  occured  at 
tk-  The  localization  phase  is  typically  a  variant  on  the  bisection  or  bracketing 
algorithms  found  in  the  classical  numerical  analysis  literature.  Once  the  event  is 
localized  the  integration  is  stopped,  and  the  transitions  occur. 

Although  this  basic  technique,  first  introduced  in  [4] ,  seems  to  work  well  for 
many  problems  there  are  several  situations  in  which  it  is  prone  to  failure.  The 
situations,  discussed  below,  are  illustrated  in  Figure  lb.  The  first  case  is  when 
the  trajectory  is  sufficiently  oscullatory  that  the  guard  has  an  even  number  of 
roots  in  the  interval  t*  G  (tk,tk+i\.  A  similar  situation  occurs  when  the  guard 
set  is  “thin”  or  has  sharp  corners.  These  two  cases  are  essentially  equivalent. 
Both  are  situations  in  which  many  of  the  most  common  detection  methods  can 
fail.  As  second  class  of  problems  for  which  the  standard  technique  fails,  consider 
the  case  when  the  right  hand  side  of  the  differential  equation  is  ill-defined  for 
some  X  such  that  g{x)  >0.  Perhaps  the  nature  of  the  system  is  such  that  model 
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is  only  valid  in  certain  regions  of  the  state  space.  Since  the  right  hand  side  of 
the  ODE  cannot  be  evaluated  at  the  new  point,  bisection  methods  cannot  be 
used  to  locate  the  root  more  precisely.  In  this  situation,  almost  all  common  event 
localization  methods  fail. 

Cellier  [4]  was  the  first  to  note  that  state  events  warrant  special  treatment 
and  advocated  the  discontinuity  locking  approach  still  used  today.  Gear  [8] 
demonstrated  the  inefficiencies  that  can  result  if  special  techniques  are  not  used. 
Carver  [3]  was  the  first  person  to  notice  that  the  rate  of  change  of  the  event 
function  along  the  flow  field  (i.e.  the  Lie  derivative)  was  a  critical  quantity  in 
event  detection.  The  idea  of  differentiating  the  guard  and  appending  it  as  an  ex¬ 
tra  state  variable  to  be  integrated  was  introduced  there  as  well.  In  each  of  these 
cases  events  were  detected  by  simply  looking  for  sign  changes  in  the  guard  after 
integrating  through  one  step.  As  a  result  they  fail  to  detect  an  event  when  there 
are  multiple  transitions  in  a  single  step.  Building  on  this  work,  Shampine  and  his 
colleagues  [12]  exploit  the  fact  that  interpolation  polynomials  can  be  generated 
for  the  guard  dynamics  and  are  able  to  correctly  identify  event  occurrences  using 
Strum  sequences  when  the  guards  are  of  polynomial  expressions  but  do  not  use 
this  information  to  select  step  sizes.  Several  similar  algorithms  for  event  detec¬ 
tion  in  differential  algebraic  equations  were  evaluated  in  [15].  These  techniques 
are  able  to  detect  multiple  transition  however  they  tend  to  be  expensive.  Most 
recently,  Park  and  Barton  [17]  combine  some  of  these  ideas  and  uses  methods 
from  interval  arithmetic  to  create  efficient  tests  to  determine  intervals  where  it 
is  possible  an  event  had  occured.  This  event  detection  method  seems  to  be  the 
most  reliable  technique  in  the  literature,  it  is  streamlined  and  well  suited  to  stiff 
problems.  However  since  all  of  the  techniques  use  the  discontinuity  locking  ap¬ 
proach,  none  of  these  provides  a  methodology  to  select  step  sizes  to  ensure  that 
the  state  never  crosses  the  event  surface;  thus  all  fail  to  localize  and  event  which 
occurs  in  the  neighborhood  of  a  model  singularity. 

The  idea  in  this  paper  is  to  develop  an  event  detection  technique  that  is 
not  vulnerable  to  these  pitfalls.  Using  an  analogy  to  control  theory  we  treat 
the  simulated  system  as  a  control  system,  the  integration  step  size  as  an  input, 
and  the  guard  as  the  output.  The  problem  is  the  to  select  a  “feedback  law”  (a 
rule  for  selecting  step  sizes)  such  that  as  the  simulation  proceeds  the  system 
approaches  the  event  surface  {g(x)  =  0)  asymptotically,  without  overshoot  ( 
g{x)  <  0  always).  Since  the  state  approaches  the  guard  asymptotically  there 
is  a  better  chance  events  are  detected  and  since  there  is  no  overshoot  there  is 
no  risk  of  crossing  a  model  singularity.  In  Section  2  we  review  Linear  Multistep 
numerical  integration  techniques  and  introduce  the  control  theoretic  concept  of 
input/output  linearization  which  our  algorithm  is  inspired  by;  in  Section  3  we 
develop  in  detail  the  ideas  used  in  the  method,  culminating  in  a  conceptual 
algorithm;  in  Section  4  we  successfully  solve  two  example  problems  which  can 
be  problematic  for  other  methods  and  discuss  some  of  the  limitations  of  the 
proposed  algorithm;  finally  in  Section  5  we  summarize  our  results  and  comment 
on  future  directions. 
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2  Key  Concepts 

In  this  section  we  review  numerical  integration  of  ordinary  differential  equations 
using  Linear  Multistep  Methods,  our  prefered  integration  method.  We  also  in¬ 
troduce  the  key  idea  behind  our  algorithm  which  draws  on  the  control  theoretic 
concept  of  input-output  linearization. 


2.1  Review:  Numerical  Integration  with  Linear  Multistep  Methods 

Given  the  system  x  =  f{x,t)  and  a:(0)  =  xq,  it  is  customary  to  denote  the 
approximate  solution  at  the  discrete  time  as  Xk  =  and  then  the  value 

of  the  time  derivative  may  be  written  as  fk  =  f{xk).  It  is  also  convention  to 
define  the  time  step  as  hk  =  tk-  4-1-  The  most  general  form  of  a  m-step  linear 
multistep  method  (LMSM)  is  where  Oj 

and  Pj  are  the  coefficients  of  the  method.  Particular  LMSM’s  differ  in  how  a 
and  p  are  selected.  LMSM’s  can  be  broadly  divided  into  two  categories:  if  /3o  =  0 
the  method  is  called  explicit,  otherwise  if  /3o  0  the  method  is  called  implicit 

Although  the  techniques  presented  here  can  be  applied  to  the  entire  class  of 
explicit  LMSM’s,  the  explicit  Adams  family  is  by  far  the  most  popular  and  will 
be  used  for  the  purposes  of  illustration.  In  such  a  method,  ao  —  1,  ai  =  — 1,  and 
aj  =  0  for  y  >  1.  The  /3j  ’s  are  then  selected  such  that  the  difference  equation 

m 

^k-^i =^k+ ^fc+1  (2) 

would  exactly  reproduce  the  analytical  solution  x(t)  if  it  were  a  polynomial  of 
order  m  or  lower.  In  general  the  accuracy  of  the  method  is  proportional  to 
(hk)"^.  The  Adams  family  of  methods  is  very  popular  due  to  their  large  region 
of  stability  and  efficiency.  See  any  numerical  analysis  text  for  further  details  [10]. 
Often  in  text  books,  values  of  P  will  be  supplied  as  constants;  however  this  is 
only  the  case  when  the  step  size  is  constant.  In  general,  /3  is  a  rational  polynomial 
function  of  the  previous  m  step  sizes,  Pj{hk, . . . ,  hk~m)-  Multistep  methods,  as 
opposed  to  Runge-Kutta  methods,  are  a  natural  choice  for  simulating  hybrid 
systems  because  the  polynomial  expressions  for  Pj  can  be  used  as  interpolants 
to  approximate  the  solution  at  off-mesh  points. 

2.2  Feedback  Linearization  Analogy  in  Continuous  Time 

One  feature  of  explicit  LMSM’s,  not  present  in  some  other  methods,  is  the  fact 
that  Xk+i  is  defined  by  a  difference  equation  which  is  affine  in  the  step  size 
hk-  This  property  allows  one  to  draw  comparison  with  nonlinear  control  systems 
which  often  are  affine  in  the  input.  Following  this  analogy  the  difference  equation 
of  the  numerical  method  would  be  the  system  dynamics,  the  step  size  is  viewed 
as  the  input  and  the  guard  function  is  considered  to  be  the  output  equation. 

For  the  purposes  of  illustrating  our  method,  let  us  imagine  for  a  moment 
that,  instead  of  belonging  to  the  set  of  positive  integers,  we  let  the  step  number, 


208  J.M.  Esposito,  V.  Kumar,  and  G.J.  Pappas 


k,  take  on  a  continuum  of  values,  k  e  [0,  oo).  Further  suppose  that  4  is  then  a 
continuous  function  of  the  real  variable  k,  denoted  by  t{k).  Naturally  it  follows 
that  we  would  then  write  x{t{k)),  and  g{x{t(k))).  Analogous  to  the  discrete  case 
we  then  find  that  the  “step  size” ,  which  is  our  input  variable,  can  be  viewed  as 
dynamics  of  the  event  function  (our  output  function)  are  then 

dk  \dx  dt  J  dk'  ^  ^ 

since  by  definition  ^  =  f{x)  this  can  be  rewritten  as, 

^  =  {Lfg)hik).  (4) 

Note  that  the  Lie  derivative,  Ljg  =  ^  •  /,  has  a  geometric  interpretation  here 
as  the  time  derivative  of  g{x)  along  trajectories  of  the  ODE. 

We  would  like  to  select  h{k)  in  such  a  way  as  to  ensure  that  g(x)  — >•  0  as 
A:  ^  oo.  This  may  be  accomplished  by  a  technique  from  nonlinear  control  theory 
called  feedback  linearization  (see  for  example  [2]).  Assuming  the  Lie  derivative 
is  non-zero,  selecting 


m  .  (5) 

and  substituting  into  eq.(4)  results  in 

do 

Tk  =  (6) 

where  7  is  some  positive  constant  to  be  selected  by  the  user.  The  solution  to  the 
ODE  is  then  g{k)  =  ^(0)  exp~'^‘^;  which  implies  g(k)  -4  0  exponentially,  as  A:  — >• 
00.  Thus,  by  judicious  selection  of  the  input,  one  may  cancel  the  nonlinearities 
and  stabilize  the  guard  dynamics.  In  terms  of  simulation,  by  selecting  the  step 
size  appropriately  using  eq.(5)  we  are  able  to  re-par ameterize  time  in  order  to 
make  the  guard  (as  a  function  of  the  step  number)  behave  as  a  linear  differential 
equation  which  has  a  stable  equilibrium  point  on  the  surface  g{x)  —  0. 


3  Simulation  Algorithm 

In  this  section  we  describe  the  ideas  used  in  our  simulation  algorithm:  methods 
for  computing  step  sizes  depending  on  the  form  of  the  guards  (Sect.  3.1-  3.3), 
computation  of  candidate  step  sizes  (Sect.  3.4),  dealing  with  boolean  combina¬ 
tions  of  guards  (Sect.  3.5),  merging  the  candidate  step  size  for  event  detection 
with  the  ideal  step  sizes  computed  for  integration  accuracy  and  other  implemen¬ 
tation  details  (Sect.  3.6  and  3.7).  Finally,  in  Section  3.8  these  ideas  are  presented 
as  a  concrete  algorithm. 
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While  Sect.  2.2  contains  a  useful  way  of  thinking  of  such  systems,  the  simu¬ 
lated  system  evolves  in  discrete  time.  For  a  linear  multistep  method  the  dynamics 
are 

m 

x(tk  +  hk+i)  =  Xk+i  =  Xk+  hk+i{J2l^jfk-j+i}  (V 

j=i 


which  implies  the  guard  dynamics  are 

m 

g{x(tk  -h  —  di  ^  Pjfk—j  +  l}  )•  (^) 

J  =  1 

Selecting  hk+i  to  produce  the  desired  behavior  is  somewhat  more  difficult  in 
discrete  time. 


3.1  Symbolic  Inverse 

In  theory,  provided  the  guard  is  an  invertible  function  (with  respect  to  time 
along  a  given  integral  curve),  we  can  select 


hk+i  — 


-Xk  +  9  ^iigjxk)) 

h 


(9) 


where  the  vector  fp  =  yielding  the  difference  equation  = 

'jgk:  which  has  the  solution  gk  =  and  converges  exponentially  to  g  =  0 
provided  0  <  7  <  1.  This  naturally  assumes  one  can  compute  the  symbolic 
inverse  of  the  guard,  g~^{hk+i),  which  is  an  unrealistic  assumption  in  practice. 


3.2  Exact  Linearization 


While  it  is  unlikely  that  one  would  have  a  symbolic  expression  for  the  inverse  of 
g{x{t))^  exact  linearization  is  possible  for  all  guards  with  Taylor  series  expansions 
of  finite  length  (i.e  polynomial  or  linear  guards).  We  illustrate  this  idea  with 
linear  guards,  since  they  can  be  used  to  model  a  wide  class  of  systems  either 
through  approximation  or  by  transforming  nonlinear  guards  to  linear  ones  using 
eq.(l).  If  our  event  function  is  of  the  form  g{x)  =  a  '  x  +  b,  where  a  £  and 
h  ^  R  are  constant  eq.(4)  becomes 


9k+l{hk+l)  —  gk 


(10) 


which  is  essentially  a  Taylor  series  expansion  in  hk+i  about  Since  ||/^  is 
simply  the  Lie  derivative  Lf^g,  select 


hk-^i  — 


(7  -  l)gfc 

h-,9 


(11) 


Polynomial  guards  can  be  handled  in  a  similar  manner,  by  calculating  and  in¬ 
verting  their  Taylor  series  expansions  in  hk+i- 


210 


J.M.  Esposito,  V.  Kumar,  and  G.J.  Pappas 


3.3  Approximate  Linearization 

If  nonlinear  guards  with  a  Taylor  series  expansion  of  infinite  length  are  not 
transformed  to  linear  guards,  an  approximate  linearization  technique  can  be 
used.  Approximations  using  a  Taylor  series  expansion  gives 

=  9{Xk)  +  Lff^ghkJrl  +  +  .  .  .  (12) 

It  is  possible  to  compute  the  inverse  of  p  as  a  function  of  h  for  the  Taylor  series 
expansion  using  a  result  due  to  Grobner  often  referred  to  as  the  Lie  series 

oo  1  1  n 

=  E  (13) 

p— 0  ^  dx  J 

While  the  result  is  defined  as  an  infinite  series,  a  finite  number  of  terms  can 
be  used  to  compute  an  approximate  linearization.  One  sided  convergence  is  no 
longer  guaranteed  since  uncanceled  terms  act  as  forcing  functions,  but  by  se¬ 
lecting  a  small  value  of  7  the  state  still  approaches  the  event  surface  slowly, 
increasing  the  likelihood  that  the  event  will  be  detected.  This  method  seems  to 
work  well  in  practice  since  h  is  typically  small  implying  that  the  higher  order 
terms  are  usually  correspondingly  small 


3.4  Computation  of  Step  Sizes 

As  mentioned  earlier,  the  ^d’s  for  the  Adams  Method  are  only  constant  in  the 
special  case  of  constant  step  size.  Since  we  are  proposing  to  adjust  the  step 
size  dynamically,  the  /?’s  in  the  above  discussion  are  not  constant,  but  rather 
are  rational  polynomial  functions  of  /ife+i*  Computing  the  correct  step  size  with 
eq.(ll),  for  example,  then  entails  finding  the  roots  of  a  polynomial  in  /ifc+i-  For 
example  in  the  case  of  two  step  Adams  method  /?i  =  {2hk)/hk^i  and  02  = 
1  —  {2hk)/hk+i.  Substituting  the  expressions  for  0  into  eq.(ll)  and  rearranging 
gives 


z  =  Roots[ahl_^_^  +  bhk+i  +  c]  (14) 

where  a  =  1/2  •  hk[dg/dx  •  {fk  +  /fc_i)J,  b  =  dg/dx  ■  fk  and  c  =  -(7  -  l)g{xk). 
Eq.(14)  must  be  solved  for  hk+i  at  every  time  step.  Similar  polynomials  can  be 
constructed  using  eq.(9)  or  eq.(13).  Various  algorithms  for  computing  the  roots 
of  polynomials  exist,  most  involve  constructing  the  companion  matrix  and  com¬ 
puting  its  eigenvalues.  In  general  the  polynomial  equation  determining  hk+i  will 
have  m  roots  (for  an  m-step  multistep  method),  however  only  positive  real  roots 
should  be  considered  as  candidates  for  event  times,  since  negative  roots  corre¬ 
spond  to  past  events,  while  complex  roots  are  physically  meaningless.  Assume  the 
positive  real  roots  have  been  ordered  from  smallest  to  largest  {ri,  ^2, . . . ,  r^}  C  z, 
then  in  the  simplest  case  of  a  single  guard,  corresponds  to  the  first  event 
and  hence  is  the  proper  choice  for  hk+i.  If  there  are  no  positive  real  roots  set 
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3.5  Boolean  Combinations  of  Guards 

In  many  realistic  system  models,  complex  guards  may  be  composed  of  several 
algebraic  inequalities  joined  or  modified  by  boolean  operators  {e.g.  polyhedrals 
or  semi-algebraic  sets).  If  the  guard  is  [g°'{x)  >  0)\l{g^{x)  >  0),  the  situation 
is  accommodated  by  computing  rj  and  rj,  the  smallest  positive  real  roots  for 
eq.(14)  using  g'"{x)  and  g^{x),  and  selecting  =  min[rj,rj]. 

In  the  case  of  {g^{x)  >  0)  f\{g^{x)  >  0),  we  compute  at  time  tk  the  sets  of 
positive  ordered  real  roots  {rf ,  rg , . .  • , }  and  {rf,  ■  • . ,  }  using  eq.(14).  Then 

1.  if  g'^{xk)  <  0  but  g^{xk)  >  0;  and  if  rf  <  rj,  let  hk+i  =  rf . 

2.  if  g^{xk)  <  0  but  g'^lxk)  >  0;  and  if  rj  <  rf ,  let  hk+i  =  rj. 

3.  if  both  g^{x)  <  0  and  g^{x)  <  0;  and  if  either  rf  <  rf  <  or  <  rf  < 
let  hk-\.i  =  r^  or  r^  respectively. 

Guards  prefaced  with  a  ^  operator  can  be  converted  to  the  standard  form  by 
changing  their  sign,  that  is  by  using  -g{x)  >  0  rather  than  g{x)  >  0. 

3.6  Final  Selection 

In  practice,  event  considerations  are  not  the  only  criteria  which  determine  the 
appropriate  step  size  to  be  used  in  simulation.  Often  the  simulation  will  specify 
some  minimum  step  size,  h^m  ,  below  which  roundoff  errors  affect  the  stability 
of  the  computation.  In  addition,  most  modern  numerical  integrators  estimate 
an  ideal  step  size  based  on  truncation  error  considerations,  herr-  The  resulting 
step  size  selected  by  our  algorithm  based  on  event  detection,  hk+i^  can  be  easily 
incorporated  into  existing  integration  algorithms  by  selecting  the  actual  step  size 
as 

h  =  max[/l];nin)  nii^( hfc-j- 1 , /lerr)  ]• 

In  this  way  the  original  accuracy  and  stability  properties  of  the  integration  al¬ 
gorithm  are  preserved. 

3.7  Termination  Criteria 

In  cases  where  the  guards  have  a  Taylor  series  expansion  of  finite  length,  7  =  0 
will  yield  exact  and  rapid  convergence  to  the  event  surface;  therefore  the  algo¬ 
rithm  should  be  terminated  when  g{xk+i)  =  0  If  the  guards  are  more  general 
nonlinear  functions  ,  exact  convergence  is  not  guaranteed.  In  such  situations, 
conservatively  selecting  0  <  7  <  1  will  cause  the  simulator  to  take  successively 
smaller  steps  toward  the  surface.  However,  selecting  7  too  large  results  in  slow 
convergence  rate  and  a  very  small  7  can  risk  overshooting  the  guard,  in  practice 
we  have  found  0.05  <  7  <  0.5  to  be  a  good  selection.  Slowing  down  the  sim¬ 
ulation  in  this  manner  has  the  effect  of  dramatically  increasing  the  chances  an 
event  will  be  properly  detected  and  may  event  be  useful  when  exact  lineariza¬ 
tion  is  possible.  Since  steps  are  taken  in  such  a  way  that  the  value  of  the  guard 
approaches  zero  asymptotically,  it  may  take  an  infinite  number  of  steps  to  reach 
zero  exactly.  Therefore  the  user  must  set  a  small  threshold  e  >  0  such  that  the 
procedure  is  terminated  when  g{x)  >  — e.  Alternatively  one  could  choose  to  stop 
the  procedure  once  the  computed  time  step  is  smaller  than  hmin- 
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3.8  Algorithm 

All  of  these  ideas  are  assembled  into  an  algorithm  and  implemented  in  Matlab. 
Given  by  the  user  upon  initialization: 

-  A  set  of  atomic  propositions  of  the  form  g^[x)  <  0,  g^[x)  <  0,  g^{x)  <  0, 
. . .  joined  or  modified  using  the  operators  Vj  Aj 

-  the  gain,  0  <  7  <  1;  and  termination  tolerance  e  >  0. 

Preprocessing 

1.  convert  any  guards  of  the  form  ~^g{x)  <  0  to  ~g{x)  <  0. 

2.  if  desired,  convert  any  nonlinear  guards  to  linear  guards,  using  the  transfor¬ 
mation  described  in  eq.(l),  by  appending  an  extra  state  variable. 

Repeat  until  termination 

Get  from  the  integration  algorithm  at  each  iteration: 

-  m  previous  derivatives  used  in  the  multistep  integration  method,  fk,fk-i, 

•  •  •  )  fk—m 

-  ideal  step  size  for  controlling  the  truncation  error, /igrr  and  minimum  allow¬ 
able  step  size,  hmin 

Main  Algorithm 

1.  for  each  atomic  proposition  g^-,  g^,  . . . ,  ...  compute  a  candidate  step  size 

using  the  appropriate  method: 

a)  symbolic  inverse  o{g^{x))~'^  given  by  user- 

Roots  [hfj3{h)+Xk  - 

b)  g'{x)  is  linear  or  has  been  converted  to  linear  form  and  Lf^g  ^0- 

Roots  -  (7  -  l)s‘(a:*:)]  =  z* 

c)  g\x)  is  a  polynomial  of  order  N  ~ 


N 


hp 


p=\ 


d)  nonlinear  g^{x)  -  compute  roots,  z\  using  Lie  series  (eq.  13). 

2.  for  each  set  of  roots  from  the  previous  step  etc.  discard  any  negative 

or  complex  roots.  If  there  are  no  positive  real  roots  for  a  given  z"^  set  /i*  =  oo; 
otherwise  sort  the  positive  real  roots  in  ascending  order  r*  =  {ri,r2, . . .  }. 

3.  Using  r®,r^,...,  recursively  compute  a  composite  step  size,r"'  for  each 
boolean  conjunction  using  the  rules  in  section  3.5. 

4.  combine  this  result  with  the  step  size  computed  in  the  integration  algorithm 
using  h  =:  max[hmin,  min(r*,  herr)] 

5.  integrate  through  one  step  of  size  h.  If  p(xfc-i-i)  >  —e  terminate;  else,  repeat 
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Example  1  Example  2 


Fig.  2.  Two  examples:  (1)  an  autonomous  robot  navigating  a  corridor;  (2)  a  planar 
two  link  manipulator  with  workspace  limitations. 

4  Examples  and  Discussion 

In  this  section  we  illustrate  the  effectiveness  of  our  algorithm  using  the  two 
examples  shown  in  Figure  2.  The  first,  controlling  a  car-like  robot,  represents 
a  situation  in  which  other  event  detection  methods  fail,  because  the  guard  set 
possesses  “sharp”  corners.  The  second,  a  planar  manipulator  with  workspace 
limitations,  illustrates  a  situation  in  which  many  event  localization  methods  fail 
due  to  a  model  singularity.  We  also  discuss  some  shortcomings  of  the  proposed 
algorithm. 

Example  1.  Consider  the  nonholonomic  cart  trying  to  navigate  an  indoor  envi¬ 
ronment  as  shown  in  Figure  2.  The  kinematic  equations  are 

X  cos{9)  0 

y  —  sin(^)  0 

_e\  L  0  ^ 

where  the  inputs  Ui  and  U2  are  the  forward  velocity  and  turning  rate.  The  details 
of  the  robot  control  problem  and  the  history  of  ui  and  U2  are  omitted  here,  but 
it  is  assumed  to  be  provided  by  a  controller.  The  goal  here  is  to  verify  the  efficacy 
of  the  controller  and  in  particular,  to  verify  that  the  robot  does  not  collide  with 
the  obstacles.  For  the  sake  of  simplicity,  we  ignore  the  physical  size  of  the  robot 
and  simply  think  of  it  as  a  point.  Thus  the  guard (s)  for  the  simulation  are  given 
by  the  equations  of  the  walls 

({y  -  0.5  >  0)  \/(x  -  3.5  >  0))  V((-3/  -  0.4  >  0)  /\(2.8  -  x  >  0)).  (17) 

Figure  3a  displays  a  situation  for  which  the  standard  algorithm  fails.  Inte¬ 
gration  points  are  computed  which  happen  to  land  just  outside  the  guard  region. 
Thus  the  simulator  detects  no  collision  when  in  fact  the  robot  has  collided  with 
the  walls,  near  the  corner  (x  =  2.8,  y  —  —0.4).  Figure  3b  illustrates  the  method 
presented  in  this  paper.  Observe  how  the  integrator  slows  down  as  it  approaches 
the  event  surface.  Note  that  in  this  example  the  gain  was  selected  in  such  a 
way  as  to  produce  a  very  gradual  slow  down,  for  the  purposes  of  illustrating  the 
technique.  In  practice,  since  the  guards  are  linear,  a  gain  of  7  =  0  could  have 
been  used  to  force  fast  convergence. 
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Fig.  3.  Simulations  of  the  mobile  robot  in  example  1:  (a)  standard  simulation  technique 
fails  to  detect  the  collision;  (b)  our  method  slows  down  as  it  approaches  the  event 
surface. 


Example  2.  Consider  the  planar  two  link  manipulator,  as  shown  in  Figure  2, 
with  the  kinematic  equations 


OJl 

.^2, 

UJ2 

(18) 


desired  (x,  y)  positions  for  the  end  point  are  fed  to  the  controller  from  a  high 
level  planner  and  the  model  is  required  to  calculate  9i  and  62  to  achieve  these 
positions.  If  the  length  of  the  proximal  link  is  /i  and  the  distal  link  is  I2,  the 
appropriate  inverse  kinematics  relation  to  compute  ^1,^2  as  a  function  of  {x,y) 
are 


9i  —  arctan  2 
O2  =  arctan  2 


~y  -X 

- 7—  : ,  - — .  :  ±  CO 

y/x^  +  -y/x2  +  ^2 

y~lism{9i)  X  — /icos(^i) 

h  ’  T2 


2/1  v'x^  +  1/2 


-0i 


(19) 

(20) 


Note  that  it  is  possible  for  the  high  level  planner  to  be  unaware  of  the  specifics  of 
the  manipulator  and  specify  (x,  y)  points  which  are  outside  the  set  of  reachable 
positions  of  the  manipulator,  in  such  cases  the  arguments  of  the  cos“^  function 
would  fall  outside  of  the  range  of  [-1, 1]  and  the  right  hand  side  of  the  differential 
equation  becomes  ill-defined.  In  this  case,  given  li  >  I2  the  guard  would  be 


(\/ + 1/2)  <  (ii  + 12))  /\(y' (x2  +  >  (^1  _  ^2))  (21) 


with  X  =  li  cos  0i  +  I2  cos(^i  -f  02), y  =  h  sin  9i  -h  I2  sin(^i  -f  ^2)- 

Figure  4a  displays  a  simulation  of  the  two  link  manipulator  attempting  to 
track  a  reference  trajectory,  which  is  a  straight  line  in  Cartesian  space.  In  this 
case  the  reference  trajectory  eventually  falls  outside  the  workspace  of  the  manip¬ 
ulator,  where  the  right  hand  side  of  the  differential  equation  becomes  complex. 
The  traditional  integrator  generates  a  point  near  the  edge  of  the  workspace  and 
its  next  point  falls  outside  the  workspace.  Because  the  vector  field  is  ill-defined 
there,  it  is  unable  to  correctly  compute  this  new  point,  nor  is  it  able  to  activate 
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Fig.  4.  Simulations  of  the  two  link  manipulator  from  example  2:  (a)  root  bracketing 
methods  cannot  be  used  since  the  vector  field  is  ill-defined  out  side  the  workspace;  (b) 
our  method  approaches  the  surface  asymptotically  without  every  requiring  a  function 
evaluation  outside  the  workspace. 


its  root  finding  algorithm  (bracketing  technique)  since  it  requires  an  initial  point 
on  each  side  of  the  guard.  The  output  of  our  algorithm  is  shown  in  Figure  4b. 
Successively  smaller  steps  are  taken  as  the  state  approaches  the  boundary  of  the 
workspace. 

Discussion.  It  should  be  said  that,  although  our  method  is  capable  of  termi¬ 
nating  the  simulation  at  4  such  that  g{xk)  =  0  exactly,  in  some  situations,  or 
coming  arbitrarily  close  to  it  in  others,  it  can  only  be  considered  accurate  insofar 
as  the  underlying  integration  method  accurately  reproduces  the  exact  solution  to 
the  differential  equation.  That  is  to  say  that  while  g(xk)  will  equal  zero  exactly, 
Xk  itself  is  not  exact  since  it  is  generated  through  an  approximation  algorithm, 
as  in  all  numerical  analysis.  Other  limitations  include: 

—  In  eq.(ll),  which  determines  the  step  size,  one  must  divide  by  the  quantity 
Lf^g.  Obviously  the  method  is  not  applicable  when  this  quantity  is  zero. 
Infact,  by  the  inverse  function  theorem,  Lf^g  =  0  implies  that  the  inverse 
of  g(t)  used  in  eq.(9)  does  not  exist.  Geometrically,  the  differential  equation 
is  flowing  purely  tangential  to  the  boundary  of  the  guard  set,  an  alternative 
method  is  required. 

-  The  method  requires  solving  for  roots  of  eq.{14)  at  every  step,  despite  the 
fact  that  that  specialized  algorithms  exist,  this  computation  can  be  a  bit 
time  consuming  for  higher  order  methods  (higher  order  polynomials).  We  feel 
that  given  the  importance  of  discrete  event  detection  in  accurate  simulation 
this  additional  effort  is  worthwhile  although  an  efficient  exclusion  test  would 
improve  the  performance. 

5  Conclusions  and  Future  Work 

It  has  been  observed  that  there  are  a  variety  of  situations  in  which  one  of  the 
most  popular  hybrid  simulation  methods  can  fail  to  properly  detect  or  localize 


216  J.M.  Esposito,  V.  Kumar,  and  G.J.  Pappas 

the  occurrence  of  discrete  events:  either  due  to  a  multiple  number  of  zero  cross¬ 
ings  within  a  single  step  or  because  of  model  singularities.  We  present  a  method 
for  detecting  discrete  events  which,  using  techniques  borrowed  from  control  the¬ 
ory,  selects  integration  step  sizes  in  such  a  way  that  the  simulation  slows  down 
as  it  approaches  a  guard.  Our  method  guarantees  that  the  simulation  will  land 
exactly  on  the  event  surface  for  any  guard  which  has  a  Taylor  series  expansion 
of  finite  length.  Given  that  any  nonlinear  guard  can  be  transformed  to  a  linear 
form,  this  technique  is  applicable  to  a  broad  class  of  systems.  Even  in  situations 
where  nonlinear  guards  have  not  been  transformed  to  the  canonical  form,  the 
method  is  still  quite  useful  in  practice.  We  show  how  to  extend  the  method 
to  complex  guards  which  are  built  up  from  many  simple  algebraic  inequalities 
using  the  boolean  operators  and,  or  and  not  In  this  way  polyhedral  or  semi- 
algebraic  guards  sets  can  be  handled.  The  technique  is  easily  used  in  combination 
with  existing  integration  algorithms  and  does  not  adversely  affect  the  underly¬ 
ing  accuracy  or  stability  of  the  numerical  integration  technique.  Ultimately  the 
framework  presented  here  will  be  coded  in  Java  (presently  written  in  Matlab) 
and  incorporated  into  the  CHARON  [16]  simulation  suite. 

While  our  method  requires  a  variable  step  size  integration  method,  it  has 
been  observed  that  when  simulating  large  systems  such  as  Automated  Highway 
Systems  with  lOOO-j-  vehicles,  traditional  variable  step  size  schemes  are  unac¬ 
ceptable  since  they  require  all  components  to  be  simulated  at  the  same  rate. 
Thus  if  only  two  of  the  vehicles  actually  necessitate  a  step  size  reduction,  the 
entire  system  must  be  slowed  down  to  the  smallest  common  step  size,  creat¬ 
ing  gross  inefficiencies.  To  address  this  problem,  we  are  currently  considering 
using  the  techniques  presented  here  in  conjunction  with  multirate  integration 
methods  such  as  those  presented  in  [7].  When  integrating  a  systems  of  ODEs, 
multirate  methods  use  a  different  step  size  for  each  component.  Thus,  when  a 
particular  component  of  the  set  of  equations  is  changing  rapidly  a  small  step  size 
may  be  used  without  unnecessarily  slowing  down  the  integration  rate  for  other 
slowly  changing  components.  Multirate  implementation  would  prevent  agents 
not  involved  in  the  event  from  being  simulated  at  an  unnecessarily  slow  rate. 
We  believe  that  these  two  techniques  complement  each  other  and  can  be  used 
to  develop  a  powerful  simulation  tool  for  multiagent  and  hierarchical  hybrid 
systems. 
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Abstract.  We  propose  a  new  technique  for  the  identification  of  discrete¬ 
time  hybrid  systems  in  the  Piece-Wise  Affine  (PWA)  form.  The  identifi¬ 
cation  algorithm  proposed  in  [10]  is  first  considered  and  then  improved 
under  various  aspects.  Measures  of  confidence  on  the  samples  are  intro¬ 
duced  and  exploited  in  order  to  improve  the  performance  of  both  the 
clustering  algorithm  used  for  classifying  the  data  and  the  final  linear 
regression  procedure.  Moreover,  clustering  is  performed  in  a  suitably  de¬ 
fined  space  that  allows  also  to  reconstruct  different  submodels  that  share 
the  same  coefficients  but  are  defined  on  different  regions. 


1  Introduction 

In  this  paper  we  address  the  problem  of  identifying  discrete-time  hybrid  sys¬ 
tems  in  the  Piece-Wise  Affine  (PWA)  form.  The  class  of  systems  admitting  a 
PWA  description  is  broad  since  PWA  systems  provide  an  equivalent  represen¬ 
tation  for  interconnections  of  linear  systems  and  finite  automata  [23],  linear 
complementarity  systems  [14]  and  hybrid  systems  in  the  Mixed  Logic  Dynami¬ 
cal  (MLD)  form  [IJ.  In  particular,  the  MLD  representation  is  suitable  to  solve, 
via  optimization  techniques,  many  analysis  and  synthesis  problems  like  model 
predictive  control  [2],  state  estimation  [9],  formal  verification  [3],  observability, 
controllability  and  stability  tests  [1,19]. 

In  Section  2  we  introduce  the  class  of  Piece-Wise  AutoRegressive  eXogenous 
(PWARX)  models  that  provide  an  input-output  description  of  PWA  systems. 
PWARX  models  are  obtained  by  partitioning  the  space  of  the  regressors  in 
a  finite  number  of  polyhedral  region  and  by  considering  an  affine  submodel 
on  each  region.  Therefore,  the  identification  problem  can  be  formulated  as  the 
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reconstruction  of  a  possibly  discontinuous  PWA  map  with  a  multi-dimensional 
domain. 

In  the  last  few  years,  the  Neural  Network  community  developed  algorithms  to 
solve  regression  problems  with  PWA  maps.  Among  them,  one  may  cite  Breiman  s 
hinging  Hyperplanes  [7]  and  multilayer  neural  networks  with  PWA  activation 
functions  [13].  However  all  such  algorithms  focus  on  the  estimation  of  a  continu¬ 
ous  PWA  function.  A  key  feature  of  PWARX  models  is  that  the  output-update 
map  can  be  discontinuous  along  the  boundary  of  the  regions.  This  is  due  to  the 
fact  that  many  logic  conditions  can  be  represented  through  discontinuities  in 
the  state-update  and  output  maps  of  a  PWA  system.  To  the  authors’  knowl¬ 
edge,  regression  with  discontinuous  PWA  maps  received  very  little  attention  so 
far.  In  [22]  an  algorithm  based  both  on  adaptive  and  competitive  learning  for  the 
on-line  identification  of  PWARX  models  was  proposed.  However,  its  performance 
strongly  depends  on  the  initialization  and  the  choice  of  the  learning  parameters. 
Off-line  procedure  for  the  reconstruction  of  special  classes  of  PWARX  models 
can  be  found  in  [16]  and  [4].  In  a  very  recent  work  [12]  a  regression  problem 
with  monodimensional  PWA  maps  was  considered  whereas  a  multilayer  neural 
networks  with  logic  gates  was  proposed  in  [21]. 

An  off-line  procedure  for  the  identification  of  general  PWA  systems  was  de¬ 
rived  by  the  authors  of  the  present  paper  in  [10],  The  main  difficulty  in  re¬ 
constructing  PWA  maps  is  that  estimation  of  the  linear  submodels  cannot  be 
separated  from  the  problem  of  classifying  the  data,  i.e.  of  assigning  each  dat- 
apoint  to  the  submodel  that  more  likely  generated  it.  In  order  to  accomplish 
both  tasks,  an  algorithm  that  exploits  the  combined  use  of  clustering  and  linear 
identification  techniques  Wcis  derived  in  [10].  The  key  idea  of  this  algorithm  lies 
in  a  procedure  that  reduces  the  problem  of  classifying  the  data  to  an  optimal 
clustering  problem. 

Optimal  clustering  is  known  to  be  computationally  prohibitive  [8],  and  the 
common  practice  is  to  resort  to  suboptimal  but  efficient  algorithms  like  i^-means 
(see  [8,11]  for  comprehensive  reviews  of  various  clustering  techniques).  However, 
all  the  classical  procedures  suffer  from  two  drawbacks:  first,  poor  initialization 
allows  the  algorithms  to  be  trapped  in  local  minima,  second,  their  performance 
may  be  compromised  by  the  presence  of  outliers.  In  this  paper,  we  propose  a  K- 
means”-like  algorithm  that  exploits  confidence  measures  on  the  points  that  have 
to  be  clustered  in  order  to  reduce  the  influence  of  outliers  and  poor  initializations. 
Moreover,  differently  from  [10],  clustering  is  not  performed  in  the  space  of  the 
model  coefficients,  but  in  an  extended  space  that  takes  also  into  account  the 
spatial  localization  of  the  models.  This  allows  to  distinguish  between  submodels 
that  share  the  same  coefficients  but  are  defined  on  different  regions. 

Once  the  data  have  been  classified,  linear  regression  can  be  used  to  compute 
the  final  submodels.  However,  pure  least  squares  are  not  the  optimal  choice  since 
they  are  sensitive  to  outliers  [15]  that  may  be  present  because  of  classification 
errors.  In  order  to  alleviate  this  shortcoming,  we  employ  weighted  least  squares, 
using  as  weights  suitably  defined  confidence  measures  on  the  datapoints.  Finally, 
in  order  to  find  the  shape  of  the  regions,  we  use,  as  in  [10],  linear  support  vector 
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machines  [24]  that  find  the  optimal  separating  hyperplanes  between  the  classified 
datapoints. 

The  various  steps  of  the  main  algorithm  and  two  illustrative  examples  are 
reported  in  Section  3.  Moreover,  in  Section  4  we  discuss  the  proposed  procedure 
highlighting  future  research  directions  and  possible  modifications  in  order  to 
estimate  also  the  number  of  submodels  and  the  model  orders  from  the  data  set. 


2  Problem  Statement 


We  consider  the  problem  of  identifying  Piecewise  AutoRegressive  eXogenous 
(PWARX)  systems  that  are  defined  relying  on  the  s  submodels 


y{k)  =  < 


r ai,iy{k  -  1)  +  ai^2y{k  -  2)  +  . . .  +  ~  ^a)  +  b[^^u{k  -  1)  + 

'  +  -  2)  +  . . .  +  bl^^u{k  -  rif,)  +  /i  +  ek 


as,iy(k  -  1)  +  -  2)  +  . . .  +  ~  ^a)  +  b'^^^u{k  -  1)  + 

+  b'^  2'^{k  -  2)  +  ...  -1-  b's,nM^  -  +  fs  +  €k 


(1) 


where  u  €  and  y  G  M  are  the  inputs  and  the  output  respectively,  fi  are 
displacements  and  are  noise  samples.  We  consider  a  simple  noise  model  by 
assuming  that  are  Gaussian,  independent  and  identically  distributed  random 
variables  with  zero  mean  and  variance  The  n-dimensional  vector  of  the  re¬ 
gressors  is  denoted  by 


x{k)  4  [y{k  -l)y{k-2)  ...  y{k  -  Ua)  u'{k  -  1)  u'{k  -2)...  u^{k  -  nj,)]' 

and  we  assume  that  the  regressors  lie  in  a  bounded  polyhedron,  called  regressor 
set  and  denoted  by  A'.  In  order  to  specify  a  PWARX  model  completely,  a  poly¬ 
hedral  partition  of  A'  is  given  and  the  switching  law  between  the  models 

is  specified  by  the  rule:  if  x{k)  €  A^,  the  z-th  dynamic  of  (1)  is  active.  When 
an  input/output  pair  {x(k),y(k))  is  such  that  x{k)  G  we  say  that  the  pair 
belongs  to  the  i-th  submodel.  As  discussed  in  [10],  one  advantage  of  PWARX 
models  is  that  it  is  possible  to  map  them  into  the  standard  state-space  form  of 
PWA  systems  by  using  classical  realization  theory.  Therefore,  all  the  tools  for 
analysis  and  synthesis  for  hybrid  systems  in  the  MLD/PWA  form  can  be  directly 
applied  to  the  identified  model. 

Throughout  this  paper  we  assume  that  N  input/output  points  {y{k),u{k)), 
k  0, . . .  ,  iV,  have  been  collected  in  the  dataset  S.  These  are  the  data  available 
for  the  identification  of  the  PWARX  model. 


Assumption  1  The  data  are  generated  from  the  PWARX  model  (1)  specified 
by  the  orders  fia,  fib}  Idle  number  of  submodels  s,  the  parameter  vectors 


1  0,1,2 


and  the  sets  A,  Aj,  z  =  1, . . .  ,  s, 


b^  2 


(2) 
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Remark  1.  If  the  data  are  generated  according  to  Assumption  1  and  ria,  nj,, 
5,  X  and  Xi,  i  =  1, . . .  ,  s,  are  known,  the  identification  problem  amounts  to 
reconstruct  the  s  ARX  submodels  in  (1)  and  this  can  be  done  by  using  standard 
algorithms  for  the  identification  of  linear  models  [17],  In  fact,  since  the  sets  Xi  are 
known,  we  can  classify  the  points  {x{k),y(k)),  i.e.,  collect  together  the  datapoints 
belonging  to  the  i-th  affine  submodel  and  use  them  for  its  identification. 

The  identification  problem  becomes  non-trivial  if  we  do  not  know  all  the 
quantities  mentioned  in  Remark  1.  As  discussed  in  [10]  a  fair  scenario  for  the 
identification  of  PWARX  models  is  given  by  the  following  Assumption, 

Assumption  2  Assumption  1  holds  and  the  number  of  submodels  s,  the  orders 
fiaj  o.'^d  the  regressor  set  X  are  known.  Moreover,  s  ~  s,  Ua  =  fia,  nb  —  nb 
and  X  =  X. 

The  number  of  models  depends  on  the  number  of  operative  conditions  in  which 
the  data  are  collected.  For  instance  one  can  know  in  advance  that  the  systems 
may  only  switch  between  a  normal  and  a  faulty  operating  condition,  i.e,,  s  =  2. 
The  assumption  that  the  model  orders  Ua  and  ni  are  known  is  less  realistic  but 
will  allow  us  to  concentrate  on  the  peculiarities  of  the  identification  of  PWARX 
models  without  introducing  the  difficulties  due  to  the  estimation  of  the  model 
orders.  The  shape  of  the  set  X  describes  the  physical  constraints  on  the  inputs 
and  the  output  of  the  system.  In  practice,  constraints  are  often  specified  on  each 
input /output  sample  or  on  each  input /output  increment  and  from  these  bounds 
it  is  easy  to  derive  the  set  X  once  the  orders  Ua  and  have  been  chosen  [10], 


3  The  Main  Algorithm 


Based  on  the  previous  discussion,  the  identification  problem  we  consider  reads 
as 


Problem  1.  Assume  that  the  data  {y{k),u{k)),  fc  =  0, . . .  ,  A,  are  generated  ac¬ 
cording  to  Assumption  1  and  that  Assumption  2  holds.  Estimate  the  partition 
A’i,  i  =  1, . . .  ,  s,  and  the  parameter  vectors 


(^i,2 


(3) 


characterizing  the  PWARX  model  (1). 

The  main  difficulty  in  solving  Problem  1  is  that  the  estimation  of  the  re¬ 
gions  Xi  cannot  be  decoupled  from  the  identification  of  each  submodel,  A  first 
algorithm  to  solve  Problem  1  was  proposed  in  [10].  Hereafter  we  summarize  this 
procedure  and  propose  modifications  that  improve  the  identification  results.  The 
underlying  rationale  will  be  illustrated  by  using  the  following  simple  example. 
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Fig.  1.  The  PWARX  system  (4)  (-)  and  the  dataset  (crosses) 


Example  1.  The  data  are  generated  by  the  PWARX  system 


2/W  =  { 


12] 
-1  0 
12] 


«(A;  -  1)  l]  +  f(fc)  if  u{k  -  1)  =  2;(A:)  e  =  [-4,  -1] 
u(k  -  1)  l]  +  e{k)  if  u(k  -  1)  =  x{k)  G  2  =  (-1,2) 
«(!:  -  1)  1  ]  +  €(fc)  if  u(k  -  1)  =  xik)  G  =  [2, 4] 


(4) 


where  s  —  3,  na  —  0,  n^,  =  1,  A'  =  [—4,4],  and  the  input  samples  u{k)  G  M  are 
generated  randomly  according  to  the  uniform  distribution  on  X. 

The  system  and  a  data  set  of  50  samples  with  noise  variance  cr^  =  0.05  are 
depicted  in  Figure  1. 

The  first  step  of  the  identification  algorithm  is  to  cluster  the  datapoints 
[x{k),y[k))  in  a  suitable  way  [10].  In  fact,  a  PWA  map  is  locally  linear.  Thus, 
small  subsets  of  points  x{k)  that  are  close  to  each  other  are  likely  to  belong  to 
the  same  region  [20].  For  each  datapoint  {x{j),y{j)),  j  =  1, . . .  ,  AT,  we  build 
a  cluster  Cj  collecting  {x{j),y{j))  and  the  c  -  1  distinct  datapoints  {x,y)  that 
satisfy 


Note  that  each  cluster  Cj  can  be  labeled  with  the  point  x(j)  so  having  a  bijective 
map  between  datapoints  and  clusters.  The  parameter  c  has  to  be  fixed  by  the 
user  and  this  is  a  knob  of  our  algorithm  that  can  be  adjusted.  Some  clusters  will 
collect  only  data  belonging  to  a  single  submodel  (for  instance  the  cluster  Ci  in 
Figure  1).  Those  clusters  will  be  referred  to  as  pure  clusters.  Clusters  collecting 
data  generated  by  different  submodels  will  be  called  mixed  clusters  (see  the 
cluster  C2  in  Figure  1). 

We  assume  that  c  >  n  so  that  we  can  identify  an  affine  model  by  using 
the  samples  contained  in  each  cluster.  For  this  purpose  every  linear  regression 
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technique  can  be  used  and  we  adopt  least  squares  estimation.  The  vector  of 
coefficients  0^^’^  estimated  from  the  data  in  Cj  is  then  computed  through  the 
well-known  formula 


0:1X2...  Xc 

1  1  ...  1 


(6) 


where  x*  are  the  vectors  of  regressors  belonging  to  Cj  and  yc^  is  the  vector  of  the 
output  samples  in  Cj.  A  classical  result  in  least  squares  theory  ensures  that  the 
estimated  vectors  of  coefficients  are  Gaussian  random  vectors  with  mean  9  . 

Moreover,  their  empirical  covariance  matrix  can  be  computed  as  [17] 


V, 


SSR. 


n  -h  1 


SSR^  =  y'c^  (/  -  J/c,  (7) 


Differently  from  the  rationale  described  in  [10],  we  also  introduce  the  scatter 
matrices  [8] 

Qj=  - 'nij)(x  -  rrij)' ,  ^  a;,  j  =  1, . . .  ,  AT  (8) 

{x,y)eCj  {^,y)€Cj 

that  measure  the  sparsity  of  the  Af-points  in  the  clusters  Cj . 

Both  V~^  and  QJ^  are  related  to  the  confidence  we  should  have  in  the  fact 
that  9j  is  derived  by  using  data  belonging  to  a  single  submodel  In  fact,  the 
covariance  of  the  based  on  pure  clusters  depends  only  on  the  noise  level 

and  is  expected  to  be  smaller  than  the  covariance  of  the  9^^'^  based  on  mixed 
clusters  [10].  The  reason  is  that,  in  the  latter  case,  we  are  fitting  with  a  single 
hyperplane  datapoints  generated  by  at  least  two  hyperplanes:  If  they  do  not 
coincide,  Vj  will  also  take  into  account  the  model  mismatch  that  increases  the 
sum  of  the  squared  residual  SSRj.  On  the  other  hand,  the  confidence  level  on 
9j  should  also  depend  on  the  sparsity  of  the  A'-points  in  the  cluster  Cj.  Indeed, 
scattered  clouds  of  A'-points  are  more  likely  to  belong  to  different  submodels 
than  dense  clouds.  Therefore  the  confidence  level  should  be  also  proportional  to 
the  “magnitude”  of  Qj^.  In  order  to  illustrate  this  point,  consider  the  scenario 
depicted  in  Figure  2  where  a  two  dimensional  set  X  (partitioned  in  three  regions) 
is  shown  together  with  the  collected  A’-datapoints. 

If  the  true  coefficient  vectors  9i  and  9z  coincide  (i.e.,  the  same  model  is 
defined  on  the  regions  A'l  and  A'3)  it  is  impossible  to  assign  a  lower  confidence 
to  9^^'"^  than  to  9^^'^  on  the  basis  of  the  matrices  Vi  and  V2  alone.  Indeed,  even 
if  C2  is  a  mixed  cluster,  there  is  no  model  mismatch.  However  it  is  expected  that 
Q-^  will  be  “larger”  than  Q2  ^  and  this  indicate  that  is  more  likely  that 
is  based  on  a  pure  cluster  than  9^^''^. 

Consider  now  the  vectors  ^j  =  [(6>^^’^)', m']',  Vj  =  1,. . .  ,iV.  Following  the 
previous  discussion  we  can  approximatively  model  ^j  as  the  realization  of  Gaus¬ 
sian  random  vectors  with  mean  ^j  and  variance 
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Fig.  2.  Clusters  in  a  two-dimensional  region  X.  Crosses:  sampled  points. 


A  scalar  measure  of  the  confidence  level  we  assign  to  the  point  is  then  given 
by 

_ _ _ 

~  V'(27r)(2".+2"..+i)det(i?i)’ 

that  is  the  peak  of  the  Gaussian  centered  in  and  with  covariance  Rj . 

If  the  data  are  corrupted  by  a  small  amount  of  noise,  if  c  is  small  enough 
and  if  the  sampling  schedule  is  “fair”  (see  the  discussion  in  Section  4),  then 
a  picture  of  the  vectors  j  =  1,...  ,iV,  should  show  s  major  clusters  and 
some  isolated  points  hereafter  referred  to  as  outliers.  In  fact  we  observe  that  if 
two  clusters  and  Cj^  are  pure  and  collect  datapoints  belonging  to  the  same 
submodel,  then  and  should  be  similar  (in  the  limit  case  of  noiseless 

data  all  such  vectors  coincide).  The  outliers  correspond  to  points  computed 
from  mixed  clusters.  However,  the  information  provided  by  the  ^-vectors  alone 
may  be  misleading,  since  it  can  also  happen  that  the  same  vector  of  coefficients 
B  characterize  submodels  defined  on  different  regions  (see  the  first  and  the  third 
submodels  in  the  Example  1).  In  this  case  the  estimated  0- vectors  collapse  into  a 
single  cluster.  The  separation  of  the  corresponding  ^-points  is  achieved  because 
of  the  vectors  mj  that  measure  the  spatial  localization  of  the  models  based  on 
different  clusters  Cj.  Since  the  models  are  defined  in  different  regions,  the  rrij 
vectors  will  be  different  even  if  the  coefficients  B^^^^  are  not.  This  fact  can  be 
noticed  by  looking  at  the  plot  in  Figure  3(a)  of  the  vectors  obtained  for 
Example  1  with  c~  6. 

Remark  2.  The  parameter  c  should  be  suitably  chosen  in  order  to  obtain  non¬ 
overlapping  clusters  in  the  ^-space.  The  optimal  value  of  c  is  always  a  trade-off 
between  two  phenomena.  Increasing  c  improves  the  estimation  of  the  Bj  coeffi¬ 
cients  based  on  pure  clusters  yielding  noise  rejection  benefits.  However,  at  the 
same  time,  a  large  c  increases  also  the  number  of  mixed-clusters  (in  fact,  for 
c  =  N  all  the  clusters  become  mixed)  and  of  the  outliers  in  the  ^-space.  For  a 
thorough  discussion  on  the  role  of  the  parameter  c  we  defer  the  reader  to  [10]. 
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The  next  step  of  the  algorithm  amounts  to  clustering  the  ^-points  into  s 
disjoint  subsets  Vi.  For  this  purpose,  in  principle,  any  clustering  algorithm  can  be 
used  (see  [8,11]  for  comprehensive  reviews)  but  the  accuracy  of  the  results  can  be 
spoiled  either  by  a  poor  initialization  (that  lets  the  algorithm  be  trapped  in  local 
minima)  or  by  the  presence  of  outliers.  In  our  case  we  can  exploit  the  measures 
of  the  confidence  on  each  ^-point  in  order  to  alleviate  these  shortcomings. 

The  clustering  technique  we  propose  is  a  variation  of  the  batch  K-means 
algorithm  [18,6,11], 

Algorithm  1 


Initialize  the  centers  fiiy  i  =  1,.,.  ,s,  and  fix  a  threshold 
e>0 

1.  compute  the  clusters  Vi  of  ^-points  that  minimize 

i=l 

2.  update  the  centers  according  to  the  formula 

fii  =  - 

3.  if  max  ll/ii  — /ii||  <  e,  Vz  =  l,,..,s,  exit,  else  set  =  fli 
and  go  to  1. 


(12) 


The  main  differences  between  Algorithm  1  and  the  classical  A"-means  are  the 
rule  (11)  for  assigning  the  vectors  to  the  clusters  Vi  and  the  formula  (12)  for 
updating  the  centers  fii  of  the  clusters.  However,  it  is  important  to  note  that 
these  modifications  do  not  spoil  the  computational  efficiency  of  AT-means. 

The  use  of  the  norms  ||  •  ||^-i  in  (11)  allows  assigning  little  influence  to  the 
^-points  based  on  mixed  clusters.  Similar  considerations  justify  the  use  of  the 
weights  Wj  in  (12).  Then,  it  is  expected  that  the  centers  will  mainly  depend 
on  the  ^-points  based  on  pure  clusters.  We  can  exploit  the  confidence  weights 
also  to  provide  a  good  initialization  of  the  centers  in  Algorithm  1.  We  suggest 
to  randomly  assign  the  vectors  to  s  sets  V^  (in  a  way  such  that  each  set 
collects  approximatively  N/s  samples)  and  to  compute  the  initial  centers  /ij  as 
the  weighted  means  of  the  elements  in  (i.e.  analogously  to  formula  (12)).  If 
the  number  of  ^-points  based  on  pure  clusters  is  much  larger  than  the  number  of 
outliers,  it  is  expected  that,  because  of  the  averaging  procedure,  the  centers  will 
be  little  influenced  by  the  outliers.  In  practice,  we  noticed  that  this  initialization 
procedure  gives  very  good  results  compared  to  other  common  strategies  like 
choosing  the  centers  randomly  from  the  A'-points  in  S.  The  result  of  Algorithm  1 
applied  to  Example  1  is  plotted  in  Figure  3(b), 
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pie  1. 


0«+ 


(b)  Clustering  of  the  vectors  $j 
with  Algorithm  1.  Clusters:  trian¬ 
gle,  diamonds,  circles.  The  crosses 
are  the  centers  of  each  cluster. 


Fig.  3.  Clustering  of  the  vectors  j  =  1, . . .  ,50 


Clustering  in  the  ^-space  allows  to  classify  the  original  datapoints  with  the 
procedure  reported  in  [lOj.  In  fact,  each  point  is  associated  to  a  single  cluster 
Cj  that  is  labeled  with  the  datapoint  Therefore  we  can  form  disjoint 

subsets  Ti,  i  =  1,. ..  of  S  according  the  following  rule;  if  e  Vi,  then 
^  The  classified  datapoints  for  the  Example  1  are  shown  in 

Figure  4. 

Since  the  original  data  are  now  classified,  it  is  possible  to  identify  the  final 
s  ARX  submodels.  More  precisely  the  i-th  submodel  is  estimated  on  the  basis 
of  the  datapoints  collected  in  the  set  !Fi.  Again  one  can  use  least  squares  to 
accomplish  this  task.  This  allows  also  checking  the  goodness  of  each  submodel  by 
estimating  the  covariance  of  the  final  parameters  9i  and  using  standard  criteria 
like  confidence  intervals.  However  one  of  the  main  drawbacks  of  least  squares 
lies  in  the  sensitivity  of  the  method  to  outliers  [15]  that  may  be  present  due  to 
classification  errors.  We  can  reduce  the  harmful  effect  of  the  outliers  by  using 
once  more  the  confidence  levels  Wj  in  the  weighted  least  squares  algorithm  [17]. 
Therefore,  each  vector  9i  is  computed  as  the  minimizer  of 

X  f  (13) 

For  Example  1  we  obtained  the  following  estimates 

9[  =  [0.9659  1.9100]  ,  =  [-0.9873  -0.0240]  ,  9'^  =  [0.9580  2.2596] 

that  provide  a  good  approximation  of  the  PWARX  system  (4). 

So  far  we  have  obtained  an  estimate  of  each  affine  submodel  of  the  PWARX 
representation.  The  final  step  is  to  look  for  the  shape  of  the  polyhedral  regions 
Xi.  To  accomplish  this  task  we  used  the  pattern  recognition  procedure  proposed 
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in  [10]  based  on  linear  support  vector  machines  and  linear  programming.  Since 
the  data  have  been  classified,  the  problem  of  estimating  the  sets  amounts  to 
a  pattern  recognition  problem  [6].  Note  that  there  is  a  hyperplane  that  separates 
the  set  A'i  from  the  set  Aj,  Vj  ^  i  because  all  the  sets  Xi  are  polyhedral  and 
convex.  We  can  estimate  such  hyperplanes  by  applying  a  linear  pattern  recogni¬ 
tion  algorithm  that  separates  the  x-points  in  Ti  from  the  x-points  in  Tj,  \/j 
The  equation  of  the  estimated  hyperplane  separating  J-i  from  Tj  is  denoted  with 
MijX  =  rriij  where  Mij  and  mij  are  matrices  of  suitable  dimensions.  Moreover, 
we  assume  that  the  points  in  Xi  belong  to  the  half-space  MijX  <  rriij  . 

Due  to  errors  in  clustering,  it  may  not  be  possible  to  find  all  the  separating 
hyperplanes.  Therefore,  the  classification  algorithm  should  look  for  the  hyper¬ 
planes  that  minimize  the  number  of  misclassified  samples.  For  the  classification 
we  used  linear  Support  Vector  Machines  [24]  because  they  are  appealing  from  a 
computational  point  of  view  (they  can  be  solved  through  Linear  or  Quadratic 
Programming)  and  they  isolate,  as  a  byproduct,  the  misclassified  samples. 

Remark  3.  Note  that  classification  errors  arise  only  when  the  sets  Ti  and  J-j 
with  j  are  not  linearly  separable.  Since  Assumption  1  holds,  this  means  that 
there  were  errors  in  the  clustering  of  the  ^-vectors.  In  other  words,  the  fact  that 
the  sets  Xi  are  polyhedral  and  convex  allows  detecting  a  posteriori  clustering 
errors  (that  are  likely  to  be  caused  by  the  ^-points  based  on  mixed  clusters  Ck). 
Then,  in  order  to  improve  the  overall  performance  of  the  algorithm,  it  is  possible 
to  remove  the  misclassified  points  {x{k),y{k))  from  the  dataset  and  repeat  the 
overall  identification  procedure  on  the  reduced  set  of  datapoints. 

In  order  to  obtain  a  description  of  the  set  Xi  in  terms  of  linear  inequalities, 
it  is  then  enough  to  consider  the  bounded  polyhedron 

[M/i...  MiM']'x<[mr,...  mLm']'.  (14) 

where  Mx  <  m  are  the  linear  inequalities  describing  X.  In  (14)  there  may  be 
redundant  constraints  that  can  be  eliminated  by  using  standard  linear  program¬ 
ming  techniques. 

For  Example  1,  the  following  estimated  sets  were  obtained 

A^i  =  [-4,-0.68],  A's^  [-0.68,2.1],  A^3- [2.1,4].  (15) 

The  error  in  detecting  the  boundary  at  -1  between  Xi  and  X2  is  due  to  the  fact 
that  the  datapoint  (—0.994,0.608)  was  misclassified.  However,  as  can  be  noticed 
by  visual  inspection  in  Figure  1,  it  is  really  hard  to  decide  if  the  datapoint 
belongs  to  the  first  or  the  second  submodel  because  the  boundary  is  a  point  of 
continuity  for  the  PWARX  system.  The  results  of  the  identification  algorithm 
are  shown  in  Figure  4. 

We  conclude  the  section  by  reporting  the  identification  results  for  a  more 
complex  PWARX  system. 
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Fig.  4.  Classified  datapoints  (triangles,  diamonds,  circles)  and  estimated  model  (-). 


Example  2.  The  data  are  generated  by  the  PWARX  system 


y{k)  = 


^  r  T  r  ' 

[0.90.20]  [l/(A:-l)«(fc-l)  1]  +  e{k) -d  x{k)  e 
'  [0.3 -0.3 -5]  [j/(A:-l)«(A:-l)  1]  +  €{k)  d  x{k)  €  X2 
0.5  0.4  2]  [j/(fc  -  1)  -  1)  1  j  +  €{k)  d  x{k)  e  X3 


(16) 


where  x{k)  =  [2/(fc  -  1)  u{k  s  =  3,  «„  =  1,  =  1,  =  [-30,40]  x 

[—40,40]  and  the  regions  A'l,  As,  X3  are  shown  in  Figure  5(a).  The  input  sam¬ 
ples  u{k)  e  R  are  generated  randomly  according  to  the  uniform  distribution  on 
[-30, 40]  and  the  variance  of  the  noise  affecting  the  output  is  =  0.2.  The 
model  and  the  dataset  of  100  samples  are  depicted  in  Figure  5(a). 

The  final  results  were  computed  (with  a  non-optimized  code)  in  11.88  s  on  a 
Pentium  II  400  running  Matlab  5.3.  The  identified  submodels  and  the  classified 
datapoints  for  c  =  9  are  shown  in  Figure  5(b).  The  estimated  coefficients  are 


e[  =  [0.9108  0.1839  0.4301]  , 

02  =  [0.2926  -0.2489  -4.0013]  , 
e'3  =  [0.4826  0.3834  2.2510]  . 


4  Discussion  and  Concluding  Remarks 

The  proposed  algorithm  is  composed  of  six  steps:  build  small  clusters  of  the 
original  data;  identify  a  parameter  vector  based  on  each  cluster;  partition  the 
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-*0 


(a)  The  true  model  and  the  data- 
point  (crosses) 


(b)  Classified  datapoints  (trian¬ 
gles,  diamonds,  circles)  and  esti¬ 
mated  model. 


Fig.  5.  The  PWARX  system  (16)  and  the  identification  results 


parameter  vectors  in  s  clusters;  classify  the  original  data;  estimate  the  s  sub¬ 
models;  estimate  the  partition  z  =  1, . . .  ,  s,  by  using  a  linear  classification 
algorithm. 

For  the  clustering  in  the  ^-space,  we  propose  a  modified  i^-means  algorithm, 
although  other  procedures  can  be  considered  to  cope  with  the  problem  of  end¬ 
ing  in  local  minima.  For  instance,  one  can  resort  to  soft  competitive  clustering 
algorithms  that  are  less  sensitive  to  initialization  [11].  In  order  to  improve  the 
performance  of  the  clustering  algorithm,  it  is  also  possible  to  exploit  the  mea¬ 
sures  of  confidence  on  the  ^-points  in  order  to  detect  the  outliers  in  the  ^-space, 
eliminate  them  from  the  set  of  the  ^-points  and  eliminate  the  corresponding  dat¬ 
apoints  from  the  clusters  J^i.  In  fact,  the  clusterization  of  the  outliers  may  have 
a  high  degree  of  uncertainty  and  classification  errors  may  spoil  the  accuracy  of 
the  final  classification  procedure. 

The  proposed  algorithm  gives  good  results  under  the  implicit  assumption 
that  the  sampling  in  the  A'-space  is  “fair”,  i.e.  that  the  input  is  persistently 
exciting  and  that  the  z-points  are  not  all  concentrated  around  the  boundary  of 
the  sets  Xi.  In  fact,  in  the  latter  case  it  may  happen  that  all  the  clusters  Cj 
become  mixed  even  if  a  large  number  of  samples  belonging  to  each  submodel 
has  been  collected.  We  point  out  that  the  problem  of  input  design  for  hybrid 
systems  is  quite  difficult  because  all  reachable  modes  have  to  be  sufficiently 
excited.  A  thorough  characterization  of  such  conditions  will  be  the  subject  of 
further  research. 

In  the  previous  Sections  we  assumed  that  the  number  of  models  s  is  given. 
If  it  is  unknown  it  should  be  estimated  from  the  dataset.  This  can  be  done  by 
replacing  the  modified  RT-means  algorithm  with  a  clustering  algorithm  where 
the  number  of  clusters  is  not  fixed  a  priori  such  as  the  Growing  Neural  Gas  [11] 
or  the  MDL-based  procedure  proposed  in  [5].  In  such  methods  the  number  of 
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clusters  is  automatically  detected.  It  is  apparent  that  once  the  ^-points  have 
been  classified,  the  remaining  steps  of  our  procedure  can  be  applied  without 
modifications. 

If  the  orders  and  nj,  are  unknown,  we  expect  that  their  under/over  estima¬ 
tion  can  be  detected  from  a  picture  of  the  coefficients  in  the  dual  space  (i.e.  the 
clusters  do  not  have  a  clear  boundary).  Under/over  parametrization  can  also  be 
detected  by  comparing  the  magnitude  of  the  final  parameter  vectors  with  their 
standard  deviation. 

Finally,  it  would  be  desirable  to  have  bounds  on  the  errors  affecting  the 
algorithm  both  in  identifying  the  submodels  and  in  detecting  the  regions. 
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Abstract.  Biological  cell  networks  exhibit  complex  combinations  of 
both  discrete  and  continuous  behaviors:  indeed,  the  dynamics  that  gov¬ 
ern  the  spatial  and  temporal  increase  or  decrease  of  protein  concentra¬ 
tion  inside  a  single  cell  are  continuous  differential  equations,  while  the 
activation  or  deactivation  of  these  continuous  dynamics  are  triggered 
by  discrete  switches  which  encode  protein  concentrations  reaching  given 
thresholds.  In  this  paper,  we  model  as  a  hybrid  system  a  striking  example 
of  this  behavior  in  a  biological  mechanism  called  Delta-Notch  signaling, 
which  is  thought  to  be  the  primary  mechanism  of  cell  differentiation 
in  a  variety  of  cell  networks.  We  present  results  in  both  simulation  and 
reachability  analysis  of  this  hybrid  system.  We  emphasize  how  the  hybrid 
system  model  is  computationally  superior  (for  both  simulation  and  anal¬ 
ysis)  to  other  nonlinear  models  in  the  literature,  without  compromising 
faithful  modeling  of  the  biological  phenomena. 


1  Introduction 

1.1  Lateral  Inhibition  and  Developmental  Biology 

The  emergence  of  differentiated  cell  types  from  an  initially  homogeneous  popula¬ 
tion  is  a  well-studied  phenomenon.  DiflFerentiation  occurs  in  all  animal  and  plant 
embryonic  tissue,  particularly  such  species  as  Drosophila  melanogaster  (fruit  fly) 
and  Xenopus  laevis  (South  African  claw- toed  frog)  have  been  extensively  stud¬ 
ied.  Genes  control  cell  fate  by  controlling  the  type  and  amount  of  proteins  made 
in  a  cell.  Proteins  in  turn  affect  gene  activity  by  turning  “on”  or  “off”  gene 
expression  thereby  affecting  the  production  of  proteins  themselves.  Hence  differ¬ 
ential  gene  activity  is  considered  the  key  to  cell  diflferentiation  (Wolpert  [1])  and 
protein  concentrations  in  a  cell  are  a  good  measure  of  gene  activity.  The  idea 
that  lateral  signaling  between  cells  through  the  Delta-Notch  protein  pathway  is 
responsible  for  some  cell  fate  decisions  has  gained  wide  acceptance. 

A  concise  description  of  the  biological  background  follows  (Lewis  [2]):  Delta 
is  a  transmembrane  protein  that  binds  and  activates  its  receptor,  the  transmem¬ 
brane  protein  Notch,  in  neighboring  cells.  The  activation  of  Notch  has  a  “direct 
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and  immediate”  effect  on  gene  expression.  Hence  Notch  signaling  directly  con¬ 
trols  switching  in  genetic  networks  and  cascades.  The  activation  of  Notch  in 
a  cell  affects  the  production  of  Notch  ligands  (i.e.  Delta)  both  in  itself  and  its 
neighbors.  In  the  classical  lateral  inhibition  case,  high  Notch  levels  inhibit  ligand 
production  in  the  cell  and  thus  a  cell  producing  more  ligands  forces  its  neigh¬ 
boring  cells  to  produce  less.  However,  Notch  signaling  can  also  be  responsible 
for  a  phenomenon  called  lateral  induction  where  activation  of  Notch  promotes 
ligand  production  and  thus  a  group  of  cells  cooperate  to  produce  uniformly  high 
amounts  of  ligand  and  Notch,  causing  all-or-none  behavior  that  promotes  sharp 
gene  expression  boundaries. 

Inter  and  intra  cellular  signaling  has  been  postulated  to  be  the  mechanism 
for  pattern  formation  in  an  incredibly  wide  range  of  organisms:  emergence  of 
ciliated  cells  Xenopus  embryonic  skin  (Marnellos  et  al[3]),  neurogenesis  in 
Drosophila  (Luthi  et  al[4]  and  Marnellos  et  al[5]),  sensory  cell  differentiation  in 
the  zebrafish  ear  (Haddon  et  al[6]),  chick  feather  array  (Crowe  et  al[7]),  wing  vein 
morphogenesis  in  Drosophila  (Huppert  et  al[8]),  etc.  An  example  of  the  distinc¬ 
tive  “salt-and-pepper”  pattern  formed  due  to  lateral  inhibition  is  the  Xenopus 
epidermal  layer  where  a  regular  set  of  ciliated  cells  form  within  a  matrix  of 
smooth  epidermal  cells  as  seen  in  Fig.  1.  Apart  from  pattern  formation.  Delta 
and  its  homologues  (Fringe,  for  example,  proposed  by  Moloney  et  al[9])  inter¬ 
act  with  Notch  (and  its  homologues)  to  produce  other  phenomenon  like  lineage 
decisions  and  boundary  formation  (Bray[10]),  as  well  as  stem  cell  function  and 
formation  of  skin  appendages  (Lewis  [2]). 


Fig.  1.  Xenopus  embryo  labeled  by  a-tubulin,  a  marker  for  ciliated  cell  precursors  seen 
as  black  dots.  Photograph  courtesy  of  P.  D.  Vize  {The  Xenopus  Molecular  Marker 
Resource,  http :  //vize222 .  zo .  utexas .  edu) 


1.2  Previous  Work:  Mathematical  Models 

Most  classical  models  (including  Turing’s[ll]  seminal  work  on  morphogenesis) 
depend  on  the  phenomenon  of  local  autocatalysis  with  lateral  inhibition  (LALI). 
These  are  grouped  (Oster[12])  as  neural  models,  diffusion-reaction  models  and 
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mechanical  models  and  produce  very  similar  results  in  spite  of  widely  differ¬ 
ent  internal  mechanics.  Though  successful  in  predicting  pattern  formation,  they 
suffer  from  two  main  drawbacks:  (a)  they  are  phenomenological  models  which 
usually  do  not  replicate  the  low-level  protein  dynamics  and  (b)  analysis  is  usu¬ 
ally  intractable  because  nonlinear  differential  (or  partial  differential)  equations 
are  involved.  Hence  they  are  restricted  to  numerical  solutions  and  predictions 
through  simulation. 

Previous  work  on  Delta-Notch  lateral  inhibitory  networks  focus  on  nonlin¬ 
ear  mathematical  models  of  the  protein  concentration  dynamics.  Both  Collier 
et  al[13]  and  Marnellos  et  al[3,5]  have  coupled  first  order  nonlinear  differential 
equations  which  govern  protein  production  and  decay.  The  nonlinearities  of  both 
their  models  derive  from  the  fact  that  the  Delta-Notch  protein  production  in  a 
cell  is  controlled  by  a  switching  function  which  depends  on  the  weighted  sum  of 
Delta-Notch  protein  levels.  The  necessity  of  including  nonlinear  sigmoid  func¬ 
tions  to  capture  this  switching  phenomenon  makes  analytical  proofs  of  stability 
intractable.  This  issue  has  been  addressed  by  Collier  [13]  by  analyzing  the  sys¬ 
tem  for  either  a  small  number  of  cells  (actually  a  pair  of  cells)  or  linearizing  the 
system  about  an  equilibrium. 

Marnellos  et  al[3,5]  do  not  focus  on  mathematical  analysis  but  stress  the 
experimental  validation  of  their  model.  The  model  proposed  by  Mjolsness[14], 
and  used  by  Marnellos,  is  an  attractive  starting  point  for  a  hybrid  model  because 
of  the  fairly  sharp  sigmoid  switching  function  and  the  introduction  of  switching 
thresholds  (not  used  by  Collier).  Weighted  interconnections  are  crucial  to  their 
model  and  the  crux  of  their  method  is  to  train  the  weights  in  a  network  to 
obtain  specific  patterns.  This  is  a  very  time-consuming  task  and  convergence 
is  not  guaranteed.  For  completeness,  the  cellular  automata  model  developed  by 
Luthi  et  al[4]  must  be  mentioned.  However  this  model  has  discretized  dynamics 
and  no  stability  or  convergence  analysis  has  been  done  for  it. 

1.3  Motivation  for  Hybrid  Model 

A  wide  range  of  cell  regulatory  and  signaling  mechanisms  seem  to  be  ideal  candi¬ 
dates  for  hybrid  systems  models.  The  physical  reasons  behind  this  include:  gene 
expressions  are  represented  by  the  existence  (or  absence)  of  certain  proteins; 
protein  concentration  dynamics  are  described  by  constant  exponential  growth 
and  decay  rates  coupled  with  discrete  switches;  protein  production  is  switched 
on  or  off  depending  on  the  expression  of  other  genes,  i.e.  presence  or  absence 
of  other  proteins  in  sufficient  concentrations;  complexity  is  introduced  by  the 
massive  interconnections  in  the  discrete  switching  circuit  and  logic  (it  is  not  un¬ 
common  to  find  complicated  repressive  and  promoter  feedback  channels  forming 
genetic  circuits,  e.g.  McAdams  and  Arkin[15]).  These  observations  suggest  that 
a  piecewise  affine  hybrid  model  would  be  a  very  good  choice  for  modeling  these 
systems.  Using  simple  continuous  dynamics  and  lumping  the  complexity  into  the 
discrete  inputs  gives  us  the  capability  (current  and  future)  to:  analyze  the  model 
mathematically  and  prove  reachability  and  convergence  for  a  wide  set  of  initial 
conditions,  extract  important  parameters  and  predict  their  effects  on  the  system 
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evolution  without  simulation,  and  suggest  biological  experiments  to  validate  the 
model  as  well  as  refine  it. 

The  validity  of  our  assumptions  in  developing  the  hybrid  model  are,  of  course, 
open  to  question  and  we  will  justify  them  as  we  go  deeper  into  model  develop¬ 
ment,  analysis  and  verification.  Our  current  research  demonstrates  the  applica¬ 
bility  of  hybrid  systems  modeling  and  analysis  to  a  potentially  limitless  field, 
that  of  cell  cycle  regulation  and  control.  This  paper  describes  our  first  steps  in 
defining  the  hybrid  automata  for  Delta-Notch  signaling  and  the  analysis  of  some 
simpler  cases  which  show  how  certain  parameters  critically  affect  the  steady 
state  behavior  of  the  system.  It  also  contains  simulation  results  and  comparison 
with  previous  nonlinear  continuous  models  which  clearly  show  that  our  hybrid 
models  faithfully  replicate  the  physical  phenomena. 


2  Model 

The  hybrid  system  model  that  we  develop  models  the  effect  of  intercellular  Delta- 
Notch  signaling  on  the  intracellular  concentrations  of  those  proteins.  The  follow¬ 
ing  properties,  based  on  experimental  data,  are  incorporated  in  the  model:  (a) 
direct  contact  between  cells  is  a  prerequisite  for  Delta-Notch  signaling  to  occur. 
Thus  only  neighboring  cells  (in  addition  to  feedback  from  the  cell  itself)  affect 
the  protein  concentration  dynamics  of  a  cell,  (b)  Notch  production  is  triggered 
by  high  Delta  levels  in  neighboring  cells,  (c)  Delta  production  is  triggered  by 
low  Notch  concentrations  in  the  same  cell,  (d)  high  Delta  concentrations  lead 
to  differentiated  cells  and  low  Delta  levels  to  undifferentiated  cells  and  (e)  both 
proteins  decay  exponentially. 

These  properties  are  fairly  orthodox  (Lewis[2])  and  are  used  in  the  model  de¬ 
veloped  by  Collier  et  al[13].  Our  model,  presented  in  the  next  section,  is  similar 
to  that  of  Marnellos[3],  with  the  exception  that  we  replace  his  continuous  sig¬ 
moid  switching  curve  for  protein  production  (and  gene  expression)  by  a  discrete 
switch  or  signum  function.  While  experimentally,  the  gene  expression  switch  is 
determined  to  be  a  fairly  steep  sigmoid,  as  shown  in  Fig.  2(b),  we  will  show  by 
comparison  with  the  nonlinear  model  that  the  signum  function  is  justified.  The 
signum  allows  us  to  model  the  system  as  a  piecewise  affine  hybrid  system  since, 
in  the  absence  of  switching,  the  continuous  dynamics  are  affine  and  consistent 
with  the  simple  constant  production  and  exponential  decay  postulated  (a  more 
accurate  model  of  the  continuous  dynamics  can  be  derived  from  chemical  kinet¬ 
ics  as  outlined  by  Tyson  et  al[16]).  The  “direct  contact”  assumption  restricts  the 
discrete  inputs  of  the  automaton  to  be  a  function  of  chemical  concentrations  in 
neighboring  cells  and  in  the  cell  itself. 

For  a  more  biologically  faithful  model,  we  have  approximated  the  sigmoid  by 
a  piecewise  linear  switching  function  (of  which  the  signum  is  a  limiting  case). 
Preliminary  analysis  shows  that  the  parameter  constraints  are  modified  from 
those  derived  with  the  signum  switch  by  a  term  related  to  the  slope  of  the 
switch.  In  the  limiting  case,  when  the  slope  of  the  switch  tends  to  infinity,  the 
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constraints  converge  to  those  for  the  signum.  However,  these  results  are  not 
discussed  here  and  will  be  the  subject  of  a  future  publication. 

The  spatial  layout  of  the  embryonic  cell  epidermal  layer  is  two  dimensional 
(planar),  in  which  the  cells  are  arranged  in  a  hexagonal  close-packed  lattice  as 
shown  in  Fig.  2(a).  The  indexing  scheme  for  each  cell  and  its  six  nearest  neighbors 
is  also  given  in  Fig.  2(a). 


(a)  Labeling  scheme  for  cells  in  two  di-  (^)  Sigmoid  switching  curve 

mensional  arrays 


Fig.  2.  Spatial  layout  and  switching  curve  of  the  model 


A  note  regarding  notation:  the  variable  naming  convention  follows  Marnellos 
et  al[3]  and  the  formal  definition  of  the  hybrid  automata  strictly  follows  the 
conventions  given  by  Tomlin[17]. 


2.1  Model  of  a  Single  Cell,  Two  Cell  and  N  X  N  Cell  Network 

Each  biological  cell  is  modeled  as  a  four  state  piecewise  affine  hybrid  automaton. 
The  four  states  capture  the  property  that  Notch  and  Delta  protein  production 
can  be  individually  switched  on  or  off  at  any  given  time.  It  is  assumed  that  there 
is  no  command- actuation  delay  in  the  mode  switching.  The  formal  definition  of 
the  hybrid  automaton  is  given  by: 


Hi  =  {Qi,Xi,Ei,Vi,Initi,  fiJnvi.Ri) 

Qi  =  te,  92, 93, 94} 

Xi  =  e  IR^ 


El  =  :  uo  =  = 

Vi=0 


Initi  =Qi  X  {Xi  €  IR^  :  vd,vn  >  0} 


/i(9,^) 


[-Xd^d',  -XnVn]^  ff  9  ==  9i 

[Rd  —  XdVd; -X^vp;]^  a  q  =  q2 
[~XdVd’,  Rn  ~  X^vpi]  ii  q  —  q^ 

[Rd  —  XdVd\ Rn  —  Xp/Vn]^  if  q  ~  q^ 
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ri,  <  ho,  UN  <  hN}}  u  {q2,  {UD  >  ho.UN  <  h^}} 

{q3,{'^D  <  ho, UN  >  hN}}^{g4,{uD  >  ho.UN  >  ^At}} 

(9I:  {UD  >hD/\UN<  Hn})  e  q2 
Ri  {qi,  {ud  <  hi)  Aun  >  hN})  E  93  X 

(91  j  {ud  ^  ho  A  UN  ^  hN})  ^  94  X  IR^ 

fej  [ud  <hD  ^  UN  <  hN})  ^  9i  X  ]R^ 

Ri  (927 {ud  <  ho  ^un  >  hN})  G  93  X  IR^ 

■Ri  (925  {ud  ^  ho  a  un  >  hN})  ^  94  X  IR^ 

Ri  (935  {ud  <  hi)  A  Un  <  Ajv})  €  9i  X  ]R^ 

Ri  (93,  {ud  >  ho  Aun  <  hN})  €  92  X  IR^ 

Ri  (93,  {ud  >  ho  Aun  >  hN})  €  94  X  IR^ 

Ri  (94,  {ud  <  hD  Aun  <  hN})  €  9i  X  IR^ 

Ri  (947  {ud  >  hD  Aun  <  hN})  G  92  X  IR^ 

Ri  (947  {ud  <  hD  a  Un  >  hN})  G  93  X  IR^ 

where,  vd  and  vn-  Delta  and  Notch  protein  concentrations,  respectively,  in  a 
cell;  v})\  Delta  protein  concentration  in  neighboring  cell;  \d  and  Ajy:  Delta 
and  Notch  protein  decay  constants  respectively;  Rd  and  Rn:  constant  Delta  and 
Notch  protein  production  rates,  respectively;  hD  and  hN'  switching  thresholds 
for  Delta  and  Notch  protein  production,  respectively.  Ro,  Rnj  ^d  and  \n  are 
experimentally-determined  constants.  The  switching  thresholds  ho  and  hN  are 
unknown  and  we  derive  possible  ranges  for  them  which  are  biologically  consis¬ 
tent.  In  the  single  cell,  v]^  —  G  {!,... 6}.  The  inputs  uo  and  un  are  the 
physical  realization  of  properties  (b)  and  (c)  of  the  model  outlined  before.  Fig. 
3(a)  shows  the  transition  diagram  for  the  hybrid  automaton  iJi,  in  which  the 
transition  labels  have  been  omitted  for  figure  clarity. 

The  two  cell  hybrid  automaton  H2  is  the  composition  of  two  single  cell  au¬ 
tomata,  to  form  a  model  with  four  continuous  states  and  16  discrete  modes. 


(a)  Transition  diagram  for  a  single  cell  Hybrid  automaton  for  a  3  x  3  array: 

automaton:  4  discrete  modes.  49  ^^^cTete  modes. 


Fig.  3.  Hybrid  systems  model  of  a  single  cell  and  a  planar  array. 
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Here,  v})  ^  0  for  each  of  the  two  cells,  and  thus  the  Delta  level  of  each  cell  is 
communicated  to  its  neighbor  to  control  Notch  production.  Modeling  the  full  two 
dimensional  layer  of  cells  involves  composing  N  x  N  single  cell  hybrid  automata 
with  interconnections  as  shown  in  Fig.  3(b).  The  simulation  results  which  follow 
are  from  this  planar  cell  array  model. 

3  Simulation  Results 

Using  the  model  defined  in  the  previous  section,  extensive  simulations  were  car¬ 
ried  out  for  different  size  cell  arrays.  In  a  biological  sample,  it  is  usually  assumed 
that  the  initial  conditions  on  protein  concentrations  are  nearly  homogeneous, 
thus  in  our  simulation  the  initial  protein  concentrations  in  the  cells  are  taken 
randomly  from  a  normal  distribution  with  unity  mean  and  a  variance  of  0.05. 
We  assume  that  the  protein  concentrations  at  the  boundary  cells  are  initially 
at  zero  (though  periodic  protein  concentrations  at  the  boundary  have  also  been 
simulated).  The  rate  constants  and  Ajv  are  set  to  unity  (the  equa¬ 

tions  are  assumed  to  be  normalized)  and  the  switching  thresholds  are  hn  =  —0.5 
and  =  0.2  which  are  in  the  range  which  produces  sensible  biological  results; 
these  we  derive  in  the  next  section. 

The  emergent  steady  state  behavior  of  a  20  x  20  network  is  shown  in  Fig. 
4(a)and  a  50  cell  loop  in  Fig.  4(b).  The  grey  cells  are  differentiated  cells  with  high 
Delta  and  low  Notch  concentrations  while  the  white  ones  have  high  Notch  and 
low  Delta  concentrations.  The  model  accurately  captures  the  salt-and-pepper 
pattern  of  the  real  biological  event. 


Undifferentiated  Cells 


(a)  400  cell  array 


(b)  50  cell  loop 


Fig.  4.  Simulation  results  showing  the  steady  state  of  each  cell.  Grey  indicates  a  dif¬ 
ferentiated  cell  and  white  indicates  an  undifferentiated  cell. 


The  key  results  from  the  simulation  runs  are:  (a)  near-regular  pattern  for¬ 
mation  emerges,  especially  for  larger  array  sizes,  (b)  each  cell  hybrid  automaton 
H\  is  bistable,  i.e.  it  converges  to  the  equilibrium  in  either  state  q2  or  ^3  and 
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stays  locked  there.  No  oscillations  were  encountered  in  the  simulations  (with  one 
exception,  discussed  later) .  This  nicely  models  the  fact  that  cells  eventually  pro¬ 
duce  either  Delta  or  Notch  proteins  but  not  both,  (c)  the  emergent  patterns  are 
very  sensitive  to  the  initial  conditions  for  small  array  sizes.  But  this  sensitivity 
decreases  as  the  array  size  increases.  This  result  is  similar  to  that  reported  by 
Collier  et  al[13],  (d)  the  steady  state  patterns  for  the  cell  network  follow  the  rules 
that:  no  two  differentiated  cells  lie  next  to  each  other  and  no  undifferentiated 
cell  can  be  completely  surrounded  by  other  undifferentiated  cells.  This  result  is 
important  from  the  biological  point  of  view  as  experiments  show  that  this  is  the 
preferred  steady  state  in  organisms.  We  show  later  that  this  result  is  dependent 
on  the  switching  threshold  values  ho  and  hpj  and  (e)  another  interesting  result 
which  emerges  from  the  simulations  is  the  following  phenomenon:  the  cell  differ¬ 
entiation  seems  to  start  at  the  boundary  and  propagates  inwards  in  the  network. 
This  might  have  biological  significance  and  is  also  reported  by  Collier  et  al[13]. 

4  Analysis 

In  this  section  we  will  analyze  the  equilibria  for  a  single  cell  and  a  two  cell 
network  by  performing  an  existence  and  reachability  (convergence)  analysis  for 
the  hybrid  automaton  in  each  case.  From  now  on,  we  define  boundary  conditions 
to  be  the  discrete  inputs  of  the  automaton  (the  Notch  protein  concentration  from 
the  same  cell,  and  Delta  protein  concentrations  from  neighboring  cells). 

4.1  Single  Cell  Hybrid  Automaton 

Proposition  1  (Existence  of  equilibria  of  Hi),  The  equilibria  (vo^'^n) 

Hi  depend  on  the  switching  threshold  ho,  dnd  are  as  given  in  Table  1. 

Proof.  We  prove  this  by  constructing  an  algebraic  test  for  the  existence  of  equi¬ 
libria  in  each  mode.  The  equilibrium  point  exists  if  and  only  if  it  satisfies  the 
constraints  defining  the  mode,  given  by  Invi.  We  substitute  the  equilibrium 
for  each  mode  into  the  corresponding  invariant  for  each  mode  which  gives  the 
condition  for  its  existence.  For  example,  for  mode  qi  the  equilibrium  is  given 
by  (t;Jj  =  =  0).  The  invariant  for  the  mode  is  {uo  <  ho,UN  <  h^}. 

Since  uo  =  ~vn,  we  substitute  uo  =  0  in  the  invariant  to  derive  the  condi¬ 
tion  0  <  ho  <  hj\[.  Similarly,  we  perform  the  computation  on  q2,Q3,(}4  to 

give  the  conditions  in  Table  1.  □ 


Table  1.  Existence  conditions  for  equilibrium  points  of  Hi 


Mode 

Equilibrium 

Existence  condition 

Label 

Qi 

Q2 

q3 

'74 

Vd  =  0,v*^  =  0 

v’d  =0,v*ff  =  ^ 
■^D  = 

0  <  ho  A  un  <  hj^f 

0  >  /iD  A  UN  <  hN 
— <  ho  A  UN  >  hN 
—  Ao  A  Un  ^  hN 

dead  cell 
differentiated  cell 

undifferentiated  cell 

“confused”  cell 
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Note  from  Table  1  that  the  existence  of  equilibria  can  be  directly  influenced  only 
through  manipulating  ho,  since  Ujv  is  an  external  input  over  which  the  cell  has 
no  direct  control.  Another  important  observation  is  that  the  constraints  on  Hd 
are  mutually  exclusive  for  modes  (^1,92)  and  From  the  biological  point 

of  view,  a  dead  (no  proteins  being  produced)  or  confused  (both  proteins  are 
being  produced)  steady  state  should  be  excluded  from  the  model.  By  restricting 
the  switching  threshold  /i^  :  <  0,  we  can  eliminate  the  equilibria  for 

modes  qi  and  ^4.  This  ensures  that  the  cell  can  only  converge  to  a  differentiated 
or  undifferentiated  steady  state  depending  on  the  environment  (acting  through 
un). 

In  the  following,  recall  that  O  means  “eventually” . 

Proposition  2  (Reachability  and  convergence  of  Hi).  If  0(un  <  h^)  V 
0(ujv  >  then  Hi  converges  to  an  equilibrium  in  either  mode  ^2  or  mode 
93- 

Proof  We  first  construct  the  pruned  transition  diagram  by  eliminating  from 
the  full  transition  diagram  of  Fig.  3(a)  the  transitions  for  each  model  which  are 
never  enabled.  For  example,  in  mode  52  >  Notch  protein  concentration  Ujv  is  ex¬ 
ponentially  decaying  and  the  invariant  implies  -ujv  >  ho .  Hence  the  transitions 

(^2,  {ud  <  ho  <  hj^})  e  gi  X IR^  and  Ri  (^2,  {ud  <  ho  Aun  >  hN})  € 
qs  X  IR^  are  never  enabled  because  the  condition  ud(=  -v^)  <  ho  is  always 
false,  where  h^  is  a  given  negative  constant.  Repeating  this  across  all  qi,  the 
pruned  transition  map  is  given  by: 


R 


1  • 


(91?  {'OD  >  ho  A  <  /ijv}) 
•^1  {qi  5  {'O'D  <  hr)  A  tijv  >  /ijv}) 
Pi  {qi,  {'O'D  >  A  U]s[  >  /ijv}) 
(^2,  {uD  >hr)Aurj>  h^}) 
Pi  {q^i  {'o>D  <  hr)  A  un  <  hri}) 

Pi  (^4,  {ud  <  hr)  Aun  <  hri}) 
Pi  (94,  {'iiD  >  hr)  A  uj\i  <  hrj}) 
.Pi  (94,  {ud  <  ho  Aun  >  /liv}) 


€  92  X  IR^‘ 
6  93  X  IR^ 
6  94  X  IR^ 
G  94  X  IR^ 
e  9i  X  IR^ 
G  9i  X  IR^ 
G  92  X  IR^ 
e  93  X  IR^. 


The  transition  diagram  drawn  in  Fig.  6(a)  represents  this  pruned  transition 
diagram,  and  reachability  and  convergence  can  be  deduced  by  tracing  executions 
through  it.  We  analyze  the  case  in  which  after  finite  time  the  boundary  condition 
UN  either  always  stays  less  than  h^  ot  always  stays  greater  than  or  equal  to 
The  continuous  dynamics  of  Hi  is  exponentially  stable  and  will  converge 
to  the  equilibrium  in  mode  92  or  mode  93  depending  on  whether  un  <  hj^  or 
UN  >  hN,  respectively,  as  shown  by  the  phase  portraits  given  in  Fig.  5.  Note 
that  the  phase  portraits  show  that  the  natural  tendency  for  an  isolated  cell  in 
vacuo  is  to  become  differentiated.  □ 


When  the  environment  is  time- varying  with  variation  outside  bounds  on  /i^v, 
there  is  no  guarantee  of  global  convergence  to  a  particular  equilibrium.  Since 
there  is  no  equilibrium  common  to  both  phase  portraits  when  un  <  hN  and 
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Fig.  5.  Phase  portrait  for  a  single  cell  hybrid  automaton 


UN  >  hN,  by  varying  un  the  environment  can  force  the  system  out  of  an  equi¬ 
librium  point.  A  particularly  diabolic  environment  could  keep  the  automaton 
cycling  through  the  modes  indefinitely.  For  example,  from  Fig.  6(a)  we  can 
identify  the  cyclic  sub-graph  •  •  •  ^  ->  ^2  — 94  “>  ^3  ^  ->  *  •  •  which 

the  environment  might  force  the  automaton  to  take  for  an  indefinite  interval  of 
time.  The  reason  for  this  behavior  lies  in  the  reductionism  involved  in  the  model. 
By  isolating  the  cell  from  a  larger  system  we  have  made  it  reactive  to  external 
inputs  but  removed  its  ability  to  influence  the  environment.  Given  that  the  en¬ 
vironment  is  largely  made  up  of  cells  like  itself,  the  lack  of  two-way  signaling 
clearly  hampers  analysis.  If  more  cells  are  explicitly  included  in  the  model,  as  we 
shall  see  in  subsequent  sections,  the  behavior  of  the  cells  are  more  predictable 
and  can  be  shown  to  be  globally  stable.  A  more  elegant  solution  is  to  try  to 
model  the  environment  as  a  “super-cell”  which  is  reactive  to  external  inputs  and 
replicates,  at  a  higher  level  of  abstraction,  the  dynamics  of  a  large  population 
of  cells.  Another  approach  might  be  to  eliminate  the  continuous  dynamics  alto¬ 
gether  and  work  with  a  discrete  transition  system.  Both  of  these  solutions  are 
subjects  of  ongoing  and  future  research. 

4.2  Two  Cell  Hybrid  Automaton 

Proposition  3  (Existence  of  equilibria  of  H2)^  Existence  of  equilibria  of 
the  continuous  dynamics  of  H2  depends  on  the  switching  thresholds  ho  und  hN, 
and  is  given  in  Table  2  for  zero  boundary  conditions. 

Proof  The  proof  is  similar  to  that  for  the  single  cell  automaton.  Each  equi¬ 
librium  must  satisfy  its  modal  invariant  which  provides  an  algebraic  test  for 
existence:  we  solve  for  the  equilibrium  of  each  mode  and  substitute  it  into 
the  modal  invariant.  For  example,  in  mode  97  the  equilibrium  is  given  by 
(xj  =  =  ^,0:3  =  0,^4  =  ^).  We  substitute  this  into  the  invariant  for 

97-  {3:^1  ^  hN,  —X2  >  ho,Xz  <  hN,  —2:4  <  ho}-  This  gives  the  required  condition 
for  existence  of  the  equilibrium  in  97:  hN  <  ^  ^ho  <  OAHn  >  OAho  > 

This  is  performed  for  all  16  modes  and  the  constraints  are  given  in  Table  2.  □ 
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Table  2.  Existence  conditions  for  equilibrium  points  of  H2  (the  composition  of  two 
single-cell  hybrid  automata).  Note;  x\  and  X2  are  Delta  and  Notch  levels  in  cell  1  and 
xs  and  X4  are  Delta  and  Notch  levels  in  cell  2 


Mode 

91 

92 

93 

94 

95 

96 

97 

98 

99 

910 

911 

912 

913 

914 

915 

916 


Equilibrium 


Existence  condition 


)  A  h2 


hj^  >  0  A  h.£j  >^0 
>1^  >  0  A  hj^  >  A  hjy  <  0 

A^O  >  OAh^  >  OAho  > 

>0A/ijy  >  -r-2.  A  /in  <  —  -r 

j^Ahjy<0/\h;^>0Ahj^>0 
/ijV  >  A  /i^  <  0 

^A/1£j  <0A/i^  >0A/i£)  >  — 

A  /id  <  0  ^  ^  <  -^ 

^f^D>-^^h^<0AhD>0 


\jSf 


<0A/i^>-^ 

^  A^jv  <  5^  A  /i£>  < 

0  -  ”  ^  A  /i;y  <  0  A  /i£,  >  0 
<  _  A  /iw  <  A  /in  < 


Comment 


unsatisfiable 

unsatisfiable 

unsatisfiable 

unsatisfiable 


unsatisfiable 

unsatisfiable 


unsatisfiable 

unsatisfiable 

unsatisfiable 

unsatisfiable 


It  can  be  seen  that  10  out  of  the  16  equilibria  cannot  exist  because  the  associated 
constraints  on  hj)  and  hj\i  are  unsatisfiable.  In  addition,  the  constraints  are 
mutually  exclusive  for  all  except  the  equilibria  for  modes  qj  and  ^iq.  These 
equilibria  represent  one  differentiated  and  one  undifferentiated  cell  exactly  and 
are  inseparable  due  to  symmetry.  Hence,  if  the  thresholds  are  selected  such  that 
the  equilibria  in  qj  and  910  exist,  all  other  equilibrium  points  are  unreachable. 
The  constraints  so  chosen  are  given  by, 

hj\[  :  <  /i£>  <  0  A  0  < 

Xd 

An  analysis  of  the  automaton  H2  was  done  regarding  the  effect  of  bound¬ 
ary  conditions  on  the  reachable  equilibria.  Using  the  same  equilibrium  analysis 
methods  used  in  previous  sections  it  was  determined  that  the  set  of  reachable 
equilibria  depend  critically  on  the  boundary  conditions,  i.e.  levels  of  protein  con¬ 
centration  in  the  environment.  If  the  switching  thresholds  hp  and  /iat  are  chosen 
so  as  to  give  a  biologically  consistent  equilibrium  for  zero  boundary  conditions, 
then  some  interesting  results  were  observed.  If  the  Delta  protein  boundary  condi¬ 
tions  for  both  cells  were  below  the  chosen  hjsj  value,  then  the  automaton  evolves 
as  if  the  cells  were  in  vacuo.  However,  if  any  one  of  the  neighboring  Delta  levels 
exceeds  the  chosen  then  the  automaton  admits  only  one  reachable  equilib¬ 
rium  which  is  that  in  which  the  cell  next  to  the  high  Delta  boundary  condition 
becomes  undifferentiated  and  the  other  cell  becomes  differentiated.  However,  if 
both  boundary  conditions  have  high  Delta  level  then  H2  has  only  one  reachable 
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equilibrium  which  is  that  in  which  both  cells  have  high  Notch  and  low  Delta 
level,  i.e.  both  cells  are  undifferentiated.  This  is  consistent  with  the  patterns 
observed  in  simulations. 

Proposition  4  (Existence  of  Zeno  state).  For  the  hybrid  automaton  H2,  clU 
executions  with  Init2  =  Q2  x  {^2  G  IR^  :  a:i  =  x^^X2  =  X4}  are  Zeno  executions 

and  (^q^x  :  q  =  =  [hN  ~  hjy  /ijv  —  a  Zeno  state. 

We  prove  that  the  state  is  Zeno  by  computing  the  execution  of  the  automaton 
with  the  given  initial  states  and  show  that  it  is  a  cyclic  transition  (•••->  -> 

qe  qi6  — >  O'!!  ->  — >■  •  •  •)  with  infinite  transitions  in  finite  time.  The  proof  has 

to  he  omitted  due  to  space  constraints.  Note  that  the  Zeno  state  is  a  vestige  of 
the  mathematical  model  and  not  observable  in  nature  due  to  noise.  Interestingly, 
this  Zeno  state  corresponds  to  a  saddle  equilibrium  in  the  nonlinear  model. 

Proposition  5  (Reachability  and  convergence  of  7^2 )•  For  zero  boundary 
conditions,  all  executions,  except  the  Zeno  execution,  of  the  two  cell  hybrid  au¬ 
tomaton  H2  eventually  converge  to  the  equilibrium  in  mode  qj  or  mode  qio. 

Proof.  The  construction  of  the  pruned  transition  diagram  follows  the  same  pro¬ 
cedure  as  that  for  the  single  cell  automaton.  Due  to  space  constraints  the  explicit 
pruning  procedure  is  omitted.  Figure  6(b)  gives  the  transition  map  for  i?2-  Con¬ 
vergence  is  deduced  by  tracing  executions  through  the  map.  The  equilibrium- 
containing  modes  are  invariant  under  continuous  flow  and  hence  have  no  escape 
transitions  (modes  ^7  and  gio  in  Fig.  6(b)).  It  can  be  shown  that  all  executions 
(except  the  Zeno  execution  indicated  by  the  dashed  grey  transitions  in  Fig.  6(b)) 
reach  one  of  the  two  equilibria.  Thus  the  automaton  is  bistable.  Note:  the  proof 
can  he  extended  to  include  boundary  conditions  by  constructing  the  pruned 
transition  maps  for  those  cases.  □ 


(a)  Pruned  transition  diagram  for  a  sin-  (b)  Transition  diagram  for  two  cell  au- 
gle  cell  hybrid  automaton  tomaton  with  zero  boundary  conditions. 

Fig.  6.  Pruned  transition  diagrams 
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4.3  N  X  N  Cell  Hybrid  Automaton 

While  we  performed  the  equilibrium  and  reachability  analysis  for  the  single  and 
two-cell  networks  by  hand  (by  enumerating  the  vector  fields  over  the  discrete 
modes),  as  we  analyze  larger  networks  of  cells  this  becomes  difficult.  However, 
because  the  continuous  dynamics  are  affine,  time-invariant,  with  diagonal  A  ma¬ 
trices  (which  admit  analytic  solutions),  the  equilibrium  and  reachability  analysis 
may  be  automated.  We  are  currently  designing  a  “model  checker”  based  on  these 
principles  for  this  specialized  system,  to  automate  these  analyses. 


5  Comparison  with  Nonlinear  Model 

The  steady  state  behavior  of  the  hybrid  model  and  the  nonlinear  models  are 
similar  in  simulation.  To  establish  our  model  on  a  firmer  base  it  is  necessary  to 
compare  it  with  one  of  the  benchmark  nonlinear  models,  that  developed  by  Col¬ 
lier  et  ai[13].  The  nonlinear  model  uses  nonlinear  differential  equations  coupled 
through  sigmoid  switching  functions.  Collier  proves  convergence  of  the  model  by 
determining  the  equilibria  of  the  system  and  then  using  a  set  of  “instantaneous” 
phase  portrait  projections  showing  the  flow  field  around  those  equilibria.  Figure 
7(a)  displays  the  flow  field  in  the  di(Delta  of  cell  l)-d2(Delta  of  cell  2)-plane 
with  two  sinks  and  a  saddle  point.  The  hybrid  model  successfully  captures  this 
phase  portrait,  as  shown  in  Fig.  7(b),  with  an  exception:  the  saddle  point  is  con¬ 
verted  to  a  Zeno  state.  The  hybrid  model  similarly  approximates  the  dynamics 
of  the  nonlinear  model  in  all  projections  of  the  state  space.  Hence  this  model 
is  as  expressive  in  simulation  as  the  benchmark  nonlinear  model,  yet  it  admits 
simpler  analysis. 


(a)  Nonlinear  model  (b)  Hybrid  systems  model 

Fig.  7.  Phase  plane  projections  for  two  cell  system  showing  equilibria.  Labels  di  and 
d2  are  the  Delta  protein  concentrations  in  cell  1  and  2  respectively. 
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6  Conclusion 

The  research  presented  in  this  paper  gives  a  glimpse  of  the  immense  opportunities 
for  hybrid  modeling  in  biological  systems.  It  presents  work  done  to  systemati¬ 
cally  model  a  well-known  intercellular  signaling  pathway  with  some  success.  The 
faithful  replication  of  biological  events  is  demonstrated  through  simulation  and 
the  validity  of  the  model  is  emphasized  by  comparison  to  a  benchmark  nonlinear 
model.  The  preliminary  analysis  of  the  model  is  promising  and  has  resulted  in 
the  identification  of  the  threshold  parameters  as  an  important  and  direct  arbiter 
of  cell  fate,  which  might  suggest  possible  experiments  in  the  future. 

Future  work  will  concentrate  on  the  development  of  an  automated  tool  for 
equilibrium  and  convergence  analysis  using  the  specific  geometric  properties  of 
this  system,  which  we  anticipate  will  lead  to  the  development  of  a  mathemat¬ 
ically  correct  discrete  abstraction  of  the  hybrid  model.  The  first  step  in  this 
direction  has  been  taken  by  mapping  out  the  transition  diagram  for  the  two  cell 
automata.  The  next  is  to  convert  it  to  a  pure  finite  automata.  If  that  analysis  is 
extended  to  higher  dimensional  systems,  we  may  reap  enormous  computational 
and  analytical  benefits  without  losing  sight  of  the  underlying  biology. 

We  are  hopeful  that  these  techniques  will  not  only  apply  to  the  specific 
example  presented  here,  but  also  to  a  wide  range  of  systems  in  which  protein 
growth  and  decay,  and  protein  interaction,  is  the  key  to  the  development  of  the 
biological  system. 
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Abstract.  This  paper  presents  a  formulation  and  solution  of  a  super¬ 
visory  control  problem  for  a  class  of  hybrid  systems  in  which  threshold¬ 
crossing  events  in  the  continuous  state  space  force  discrete-state  tran¬ 
sitions.  The  continuous  dynamics  are  in  turn  determined  by  a  discrete 
condition  determined  by  the  current  discrete  state  of  the  system.  The 
problem  is  to  construct  a  supervisor  that  restricts  the  discrete-state  tran¬ 
sitions  in  the  hybrid  system  so  that  the  possible  sequences  of  threshold 
events  are  contained  in  a  given  set  of  sequences  (the  desired  threshold 
event  language  of  the  closed- loop  system).  Formally,  the  hybrid  system 
supervisor  can  be  synthesized  using  the  theory  of  supervisor  synthesis  for 
discrete  event  systems.  This  procedure  is  described,  and  a  computational 
approach  to  solve  the  problem  is  illustrated  with  an  example. 


1  Introduction 

Several  types  of  control  problems  can  be  formulated  for  hybrid  systems.  In  this 
paper,  we  consider  the  problem  of  synthesizing  a  supervisor  that  restricts  the 
selection  of  the  continuous  dynamics  in  the  hybrid  system  so  that  the  sequence 
of  output  events  (generated  when  the  continuous  state  crosses  specified  thresh¬ 
olds)  is  contained  within  a  given  set  of  sequences  (the  desired  threshold  event 
language).  This  is  a  generalization  of  the  problem  considered  in  [1]  where  the  hy¬ 
brid  plant  contained  only  continuous-state  dynamics.  Similar  supervisory  control 
problems  were  also  considered  in  [2,3]  for  systems  with  discrete-time  continuous- 
state  dynamics,  and  in  [4]  for  synthesis  of  discrete  event  supervisors  for  contin¬ 
uous  and  discrete  time  systems.  In  this  paper,  the  hybrid  plant  to  be  controlled 
includes  both  continuous-state  and  discrete-state  dynamics  in  continuous  time. 

The  presentation  of  this  paper  is  organized  as  follows.  The  problem  is  de¬ 
veloped  in  Sect.  2  using  the  formalism  of  condition/event  (C/E)  systems  [5], 
C/E  systems  provide  a  framework  for  defining  continuous-time  systems  as  the 
interconnection  of  subsystems  with  discrete- valued  input  and  output  signals. 
Condition  signals  are  piecewise  constant,  whereas  event  signals  assume  non- null 
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values  only  at  isolated  instants  of  time.  The  language  for  a  C/E  system  is  de¬ 
fined  by  the  sequences  of  the  values  of  the  input-output  signals  recorded  at  their 
points  of  discontinuity. 

Section  3  describes  how  the  supervisor  synthesis  problem  for  the  hybrid  sys¬ 
tem  can  be  solved,  at  least  formally,  using  the  theory  of  supervisory  control 
for  discrete  event  systems  (DESs).  The  difficulty  in  computing  and  constructing 
the  supervisor  arises  fi:om  the  fact  that  in  general  there  may  not  be  finite-state 
generators  for  languages  derived  from  C/E  systems.  This  problem  can  be  dealt 
with  using  a  finite-state  generator  for  conservative  (outer)  approximations  to  the 
language  for  the  DES  plant,  as  described  in  Sect.  4.  The  procedure  is  illustrated 
by  an  example  in  Sect.  5.  The  concluding  section  summarizes  the  contribution 
of  this  paper. 


2  Problem  Formulation 

Consider  the  class  of  hybrid  systems  illustrated  by  Fig.  1.  The  hybrid  plant  % 
is  composed  by  interconnection  of  a  continuous  dynamic  subsystem  He  and  a 
discrete  dynamic  subsystem  The  input  signal  to  the  continuous  subsystem 


Fig.  1.  Hybrid  plant. 


'He  is  a  piecewise  constant,  right  continuous,  condition  signal  w(-),  taking  on 
values  on  a  finite  set  of  conditions  U  [5].  The  space  of  all  condition  signals  w(-) 
for  [0,  oo)  is  denoted  by  U.  The  continuous  dynamic  is  defined  by  the  continuous 
state  trajectory  x{-)  that  evolves  in  X  =  At  each  instant  t,  the  continuous 
state  trajectory  satisfies  the  differential  equation  x{t)  =  selected  by 

the  input  condition  u{t),  where  for  all  u  eU,  The  set  of  possible 

initial  values  of  the  state  trajectory  is  Xq  C  The  set  of  all  possible  trajec¬ 
tories  for  a  given  input  signal  ti(-)  €U  starting  from  any  state  in  a  set  X'  C  X 
is  denoted  by  The  function  p  :  X  — >  generates  the  continuous 

output  signal  ?/(•)  from  the  state  trajectory.  Each  component  of  y(-)  is  compared 
to  a  threshold  defined  by  a  threshold  vector  T  €  and  the  event  output  signal 
is  generated  by  a  zero  detector,  defined  for  each  component  of  the  output  signal 


Supervision  of  Event-Driven  Hybrid  Systems:  Modeling  and  Synthesis  249 


y(‘)  as  ^;(0)  =  tfo  and  for  t  >  0  and  1  <  t  <  m: 


Vii^)  ~  T,  -  0  A  >  0)(V5  e  (0,  A))  :  yi{t  -  5)  ~  T,  <  0 
^  1^0,  otherwise  ^ 

Fot  each  instant  where  any  v{t)  ^0  it  is  said  that  a  threshold  event  occurs,  oth¬ 
erwise  it  is  assumed  a  null  event  occurrence.  Let  uq  be  the  initialization  event,  an 
event  which  occurs  only  once  at  t  —  0,  and  is  associated  to  the  nondeterministic 
choice  of  the  initial  state  x(0).  Thus,  the  threshold  event  signal  t;(-)  assumes 
values  over  the  set  V  —  {0, 1}”^  U  {uq}  at  isolated  points  of  time,  and  the  space 
of  all  threshold  event  signals  t;(-)  in  [0,  oo)  is  denoted  by  V. 

The  input  signal  to  is  determined  by  the  discrete  subsystem  Hd-  The 
system  Hd  is  a  purely  discrete  dynamic  system  which  maps  nondeterministically 
event  signals  t;(-)  G  V  into  condition  signals  ti(*)  G  U,  The  feedback  of  event 
signals  from  l-Lc  to  Hd  models  physical  constraints  of  the  continuous  subsystem 
which  restricts  the  range  of  allowable  input  signals.  It  is  assumed  that  lid  can 
change  the  input  signal  if  and  only  if  a  threshold  event  is  observed.  It  is  also 
assumed  that  the  feedback  on  Fig.  1  doesn’t  lead  to  chattering,  which  means 
that  on  any  finite  interval  of  time  there  are  at  most  a  finite  number  of  threshold 
events. 

The  hybrid  plant  is  modeled  as  a  Condition/Event  (C/E)  system  in  the  sense 
of  [5]  as  follows.  The  continuous  subsystem  lie  is  defined  as  a  subset  oiV 
the  time  synchronous  cross  product  of  V  and  U,  the  set  of  all  pairs  {v{-),u{-)) 
such  that  discontinuities  in  w(-)  occur  only  at  instants  when  u(-)  is  nonzero.  The 
pair  (u(-),  w(-))  G  He  if  and  only  if  there  exists  a  state  trajectory  a:(*)  G  Xu(.){Xo) 
such  that  the  resulting  event  signal  is  u(-). 

We  introduce  the  discrete  trace  representation  for  He  as  the  4-tuple 
(W,  /,  h,  Wq)  described  as  follows.  A  piecewise  constant,  right  continuous,  condi¬ 
tion  signal  u;(-)  taking  on  values  on  W  =  and  with  initial  values  in  Wq  —  Xq, 
records  the  value  of  the  corresponding  state  trajectory  at  instants  of  discontinu¬ 
ity  in  ^  The  transition  function  f  :W  xU  -^W  for  w{')  is  such 

that 


f{w{t  ),u{t  )) 


'  w{t  ))  if  for  some  i,  1  <  i  <  m 

9i{^uit~){l^,w{t-)))  -Ti  =  0  and 

.  {3A>0){y5£{0,A)):  (2) 

w(*“)))  -Ti<0 

^  w{t~)  otherwise. 


where  3:(to))  is  the  solution  of  the  differential  equation  x  =  /u(x)  for  u  €U, 
t  >  to  and  initial  value  x{to).  The  event  output  function  h  :  W  x  W  V  is 
defined  as 


v{t)  =  h{w{t  ),w{t))  (3) 

which  outputs  the  corresponding  threshold  event  at  the  instant  of  the  state 
transition  of  w{'),  and  is  null  at  any  other  time.  This  discrete  trace  model  is 
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similar  to  the  discrete  state  model  of  [5]  except  for  the  existence  of  an  infinite 
and  uncountable  set  of  states. 

Similarly,  %  C  and  its  discrete  state  model  is  {Q,S,(t),qo),  where  Q 

is  the  discrete  state  set,  countable  and  possibly  infinite,  qq  =  q{0~)  is  the  initial 
state,  and 

q(t)  e  5{q{t-),v(t))  .  . 

u{t)  =  <t>{g{t)) 

are  the  state  transition  and  condition  output  functions  [5]. 

The  hybrid  plant  H  CV  is  obtained  by  the  cascade  and  feedback  con¬ 
nection  of  Jid  and  Tic  following  [5]. 

Consider  now  the  supervisory  control  scheme  for  the  hybrid  plant  shown  in 
Fig.  2.  The  supervisor  S  applies  a  control  input  to  the  discrete  subsystem  Hd 


Fig.  2.  Supervisory  Control  scheme  for  Hybrid  Plant. 


of  the  hybrid  plant  Ti  to  restrict  the  range  of  possible  input  conditions  to  the 
continuous  subsystem  Hc’  The  control  input  to  the  controlled  discrete  subsystem 
is  an  event  signal  m(-)  G  M  taking  on  values  on  M  =  2^,  and  is  interpreted 
as  the  set  of  allowed  conditions  to  be  chosen  by  Hd-  It  is  assumed  that  the 
supervisor  applies  a  control  input  if  and  only  if  a  threshold  event  is  observed, 
which  makes  the  discontinuities  in  m(-)  and  t;(-)  synchronous.  At  the  occurrence 
of  the  event  v{t)  G  V^,  a  control  input  m{t)  Q  U  is  applied  by  the  supervisor,  and 
if  Tid  is  at  state  the  set  of  next  possible  input  conditions  is  constrained 

to  m{t)  n  (/){S{q{t~),v(t))).  It  is  assumed  that  any  input  condition  to  Tid  can  be 
disabled,  but  the  supervisor  control  action  cannot  disable  all  possible  conditions 
for  a  given  event  v. 

The  controlled  discrete  subsystem  is  now  defined  asHd  Q  where 

the  only  difference  in  the  discrete  state  model  from  the  uncontrolled  version  is 
the  transition  function,  defined  as: 

6{q{t-),  v{t),  m(t))  =  {q{t)  G  v{t))  :  (l){q{t))  G  m(t)}  (5) 

The  controlled  hybrid  plant  is  also  given  by^  C  V  (S>  M  obtained  by 
interconnection  of  l-lc  and  7^^,  which  incorporates  the  infiuence  of  m  in  ?i. 
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The  C/E  supervisor  is  a  deterministic  C/E  system  <S  C  V  0  0  2V,  whose 

discrete  state  model  is  (Z,^,'0,zo),  where  Z  is  the  discrete  state  set  of  the  su¬ 
pervisor,  zq  =  is  the  initial  state,  and 

z{t)  =^{z(t-),u(t-),v(t)) 
m(t)  =  f{z{t~),z{t)) 

are  the  state  transition  and  event  output  functions. 

The  closed  loop  C/E  system  is  S/n  C  V,  built  by  cascade  and  feedback 
connection  of  S  and  l-L,  following  [5]. 

In  order  to  introduce  our  supervisory  control  problem,  we  express  the 
discrete  behavior  of  the  systems  in  terms  of  a  language.  Given  the  C/E  system 
V  the  language  of  P,  denoted  by  C{V),  is  the  prefix  closure  over  the  finite 
length  strings  of  records  of  the  values  of  the  input/output  signals  at  the  point 
of  discontinuities.  We  consider  the  following  problem. 

Supervisor  Synthesis  for  Hybrid  Systems  (SSHS).  Given  H  (controlled 
hybrid  plant)  and  V*  (specifications),  find  a  C/E  supervisor  S  such  that 

A  C  CiS/n)  c  E. 

3  DES  Approach 

In  this  section,  the  SSHS  is  translated  to  a  purely  discrete  event  control  frame¬ 
work,  and  a  solution  is  proposed.  The  procedure  of  this  section  is  purely  formal 
and  conceptual,  since  the  state  space  of  the  models  may  be  infinite.  Finite-state 
practical  approaches  will  be  subject  of  the  next  section. 

The  DES  model  for  the  hybrid  plant  is  a  prefix  closed  language  L  and  a 
control  structure  E.  The  language  is  defined  as  L  =  C{ii)  C  (F  x  U)*.  The 
control  structure  is  a  map  E  :  L  2^  ,  such  that  for  all  s  €  L,  E(s)  C  2^^^, 
and  7  e  E{s)  is  such  that  Vu  e  Vl{s)  =  {v  €  V  :  (3u  e  U)s  o  vu  e  L},  it  is 
always  true  that  0  C  {w  €  2/  '  vu  G  7}  0  G  U  :  s  o  vu  G  L}.  The  control 
structure  captures  the  idea  that  for  each  active  event  v  G  V  the  supervisor  may 
enable  any  nonempty  possibility  of  it  G  27  that  can  be  selected  by  the  discrete 
subsystem  for  a  give  event  v  G  V.  The  following  proposition  states  the  logical 
equivalence  of  the  controlled  hybrid  plant  and  the  DES  model. 

Proposition  1.  The  DES  model  for  the  hybrid  plant  (L,  E)  and  the  C/E  model 
C  V  0  0  2V  are  logically  equivalent,  in  the  sense  that: 

p  \/yj  =  viTniUi  o  . . .  o  VkUikUk  G  {V  X  M  X  27)*;  w  G  if  and  only  if 

s  =  viui  o  ..  .0  VkUk  G  L,  and 

2.  Mw  =  v\miUi  o  . . .  o  VkTUkUk  G  jC(7V)  and  a  =  vmu  G  V  xMxU,woaG 
C{'H)  if  and  only  if  for  s  —  I'liti  o  ...  o  VkUk  G  L,  3'y  G  E{s)  such  that 
m  =  {it  G  27  :vuG  7}. 

The  DES  supervisor  for  the  hybrid  plant  is  a  map  /  :  L  2^^^,  such 
that  for  s  G  L,  f{s)  G  E{s).  The  DES  supervisor  is  represented  by  a  state 
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machine  F  —  [P^V  x  U.p.po)  where  P  is  the  set  of  states,  po  is  the  initial 
state,  and  the  transition  function  p  :  P  x  V  x  U  — ^  P  is  such  that  for  p  ^  P 
and  a  G  V  X  U,  p{p,  a)  is  defined  if  and  only  \f  a  =  f{s)  and  s  e  (V  x  U)* 
is  such  that  p{po:S)  =  p,  where  p  stands  for  the  extension  of  the  transition 
function  to  strings  in  (V  x  U)*.  Thus,  the  supervisor  control  action  is  implicit 
in  the  machine  transitions.  We  introduce  a  formal  procedure  to  get  the  C/E 
supervisor  5,  given  by  zq),  logically  equivalent  to  the  DES  supervisor. 

The  procedure  exploits  the  state  machine  representation  of  the  C/E  supervisor, 
given  by  ((Z  x  U)  U  {zo},V  x  M  x  where  the  transition  function 

:  {{ZxU)U{zo})x{VxMxU)  {{ZxU)U{zq})  is  such  that  1(5)  =  £(<S) 
[5]. 

Consider  the  state  machines  representing  the  DES  supervisor  and  the  corre¬ 
sponding  C/E  supervisor  in  Fig.  3.  Assume  that  each  state  in  the  DES  supervisor 


Fig.  3.  Sample  correspondence  between  DES  supervisor  (right)  and  C/E  supervisor 
(left). 


has  a  unique  value  for  the  input  condition  signal  associated  with  it  on  p’s  incom¬ 
ing  arcs  in  the  DES  supervisor,  such  that  for  a  DES  state  p,  let  u{p)  denote  such 
value.  Each  state  2:  in  the  C/E  supervisor  is  associated  with  a  set  of  states  in 
the  DES  supervisor,  let’s  denote  the  set  by  P{z).  Let  V{p)  be  the  set  of  events 
on  arcs  leaving  state  p,  and  for  v£V{p),  let  U{v,p)  be  the  set  of  all  condition 
values  on  the  arcs  labeled  vu. 

It  can  be  proved  that  the  previous  algorithm  terminates  in  a  finite  number 
of  steps,  and  the  following  proposition  states  the  logical  equivalence  of  the  DES 
supervisor  and  C/E  supervisor. 


Proposition  2.  The  C/E  supervisor  S  CV  ®  M<^U  obtained  by  Alg.  1  is 
logically  equivalent  to  the  DES  supervisor  f  in  the  sense  that  Vs  =  -yiui  o  . . .  o 
^k'^k  ^  (F  X  U)*  and  a  —  vu  G  V  x  U ,  a  G  f{s)  if  and  only  if  =  ViTUiUi  o 
. .  .ovkTnkUk  G  {V  xM  X  U)*  and  a  =  vmu  G  V  x  M  xU  such  thatwoa  G  C{S) 
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Algorithm  1  Recursion  to  construct  a  C/E  supervisor  from  the  DES  supervisor 
Z  {zo};  P{Zo)  4-  {po};  ^ 

^  =  0;  -0  0;  {Sets  to  store  the  state  transition  and  output  information} 

while  Z'  0  do 

Select  and  remove  2  from  Z' 
for  p  6  P{z)  A  V  6  V’(p)  do 
m  ^  U(p,  i;); 

p'  ^  Up{pyVu),  u  G  m;  (Set  of  states  reached  by  arcs  with  v} 
if  For  all  z€Z,P'  ^  P{z)  then 

Add  new  state  z'  to  Z  and  Z'  with  P{z')  =  P' 

end  if 

Select  z'  from  Z  such  that  P{z')  =  P' 

^{z,u(p),v)  ^  z'- 
Ipiz.z')  ^  m; 

end  for 

end  while  _ _ 


The  closed  behavior  in  the  DES  framework  C  V*  is  defined  recursively 


1.  €  G  j  and 

2.  \/t  =  vio  ...ovk  £  V*  and  \/v  e  V,  t  o  V  e  a  and  only  if  t  G  A 
35  =  Viui  0...0  VkUk  e  {V  X  uy  and  3(t  =  vu£V  xU  such  that  soa  £  L 
and  a  £  f{s). 

Proposition  3.  =  C{S/'H). 

Proposition  3  indicates  that  the  SSHS  can  be  solved  by  solving  an  equivalent 
DES  supervisory  control  problem. 

SCP  Given  a  hybrid  plant  with  control  input  represented  by  a  pair  (L,  P)  and 
C  P*  (specification  languages),  find  a  supervisor  /  such  that  AC  C  E. 

Given  the  language  L  C  (V  x  U)*  and  the  language  E  C  V* ,  E  is  said  to  ^ 
controllable  with  respect  to  L,  or  just  controllable,  if  for  alH  =  vi  o  . . .  o  G  E, 
exists  5  =  Viui  o  ...  o  VkUk  £  L,  where  u\  o  . . .  o  Uk  £U*  ^  such  that 

VE{t)  =  Vl(s)  (7) 

A  threshold  crossing  event  language  is  controllable  if  the  plant  can  follow  its 
prefix  by  applying  determined  sequences  of  conditions.  This  definition  of  con¬ 
trollable  language  is  consistent  with  the  existence  of  a  control  structure  P  as 
defined  above.  For  example,  consider  the  language  L  =  [viUi  o  {v2Ui  -j-  ^2^3)]  + 
[viU2  o  (U3W1  4-  V2U2)\>  The  language  =  Vi  o  V2  is  controllable  with  respect  L 
and  the  language  E2  —  V10  vs  is  not  controllable  with  respect  to  L. 

Since  it  can  be  proved  that  the  control  structure  P  for  L  is  closed  for  the 
union  for  each  s  G  L,  then  for  E  CV*,  the  class  of  controllable  sublanguages 
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O(E')  is  nonempty  and  closed  for  the  union,  and  has  a  unique  supremal  element, 
the  maximal  controllable  language  supC(£:).  Thus,  the  following  results  bring 
a  formal  solution  for  the  SCP  [6] . 

Theorem  1.  Given  a  hybrid  plant  with  control  input  represented  by  {L,r)  and 
a  specification  language  E  C  V* ,  there  is  a  supervisor  f  such  that  =  E  if 
and  only  if  E  is  controllable  and  prefix  closed. 

Theorem  2.  SCP  is  solvable  if  and  only  if  sup  C{E)  D  A. 

The  supervisor  which  implements  sup  C(E)  is  the  optimal  solution  of  the 
SCP,  in  the  sense  of  being  minimally  restrictive.  Finally,  from  the  development 
of  this  section,  we  state  a  solution  for  the  SSHS  as  follows. 

Corollary  1.  The  SSHS  is  solvable  if  and  only  if  the  equivalent  SCP  is  solv¬ 
able.  Furthermore,  given  the  DES  supervisor  f  as  a  solution  for  SCP,  the  C/E 
supervisor  S  obtained  by  Alg.  1  is  the  corresponding  solution  for  the  SSHS. 


4  Finite  State  Approximations  for  the  HS  DES  Plant 

In  this  section,  finite  state  approximations  are  proposed  to  find  a  computable 
solution  for  the  SSHS. 

Suppose  there  is  a  finite  state  machine  H  describing  the  logical  behavior  of 
the  hybrid  plant  H,  i.e.,  L(H)  C  (V  x  U)* .  Then,  for  a  specification  language 
E  CV*,  the  language  sup  C{E)  can  be  computed  in  a  finite  set  of  steps  by  an 
algorithm  of  polynomial  complexity  in  the  number  of  states  of  both  H  and  the 
corresponding  representation  for  E  [6] . 

The  state  space  of  the  state  machine  H  is  possibly  infinite.  This  is  one  of 
the  main  problems  in  hybrid  systems  theory  since,  in  general,  the  convergence  of 
algorithms  involving  state  models  is  only  guaranteed  over  finite  spaces.  Recent 
approaches,  e.g.  [7],  propose  the  use  of  finite  conservative  approximations  for  the 
behavior  of  the  hybrid  system  to  solve  verification  problems.  In  the  context  of 
synthesis  of  supervisors  Cury  et  al.  [1,8]  show  that,  given  a  conservative  (finite) 
approximation  H'  of  H,  i.e.,  such  that  L(H)  C  L{H'),  it  is  verified  L{H'y  C 
E  then  also  holds  L{H)^  G  /,  In  other  words,  a  supervisor  solution  for  the 
approximation  H\  is  also  a  solution  for  the  original  problem,  since  the  desired 
containment  relation  L{H)^  C  E  is  preserved.  Thus,  a  supervisor  solution  for 
the  approximation  H',  is  also  a  solution  for  the  original  problem. 

Given  a  conservative  approximation  H'  of  the  behavior  of  a  plant  H  with  in¬ 
finite  state  space,  and  a  language  specifying  the  target  behavior  E,  the  synthesis 
procedure  is  applied  over  A  and  E.  If  there  is  no  solution  for  this  problem  this 
means  that  the  approximation  H'  is  too  coarse  or  that  the  specification  E  is  too 
restrictive  (it  can  not  be  satisfied  no  matter  how  accurate  the  approximation  is) 
and  needs  to  be  relaxed.  Assuming  the  specification  can  be  met,  a  refinement  of 
the  approximation  is  indicated  in  [8] ,  such  that  another  conservative  approxima¬ 
tion  can  be  computed  and  the  process  can  be  repeated  repeated  until  a  solution 
is  found. 
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5  Example 

This  section  presents  an  example  of  the  class  of  hybrid  systems  under  consider¬ 
ation.  By  this  example,  the  computational  approach  for  the  supervisor  synthesis 
for  hybrid  systems  will  be  illustrated. 


Fig.  4.  Trains  example. 


The  system  consists  of  two  trains  over  cyclic  tracks  sharing  a  piece  of  track 
(Fig.  4).  The  trains  can  travel  at  two  speeds:  fast  or  slow.  There  are  sensors  over 
the  tracks  that  register  the  crossing  of  the  trains  at  the  locations  A,  B,  C  and  D 
which,  in  turn,  correspond  to  the  events  associated  to  train  1;  similarly,  E,  F  and 
G  are  the  events  indicating  crossings  of  train  2.  At  the  instant  of  the  occurrence 
of  any  event,  each  train  can  accept  speed  change  commands.  The  slow  mode  can 
only  be  issued  between  the  locations  C-D  (for  train  1)  and  G-E  (for  train  2), 
as  indicated  by  the  gray  shade  in  Fig.  4.  The  problem  is  to  guarantee  mutual 
exclusion  on  the  shared  track,  between  the  locations  A-B  and  E-F. 

In  order  to  solve  this  supervisory  control  problem,  the  following  procedure 
was  applied,  whose  steps  are  detailed  in  the  following. 

1.  Build  the  Hybrid  System  model  in  CheckMate; 

2.  Generation  of  the  finite  state  machine  approximation  by  CheckMate,  and 

3.  Synthesis  of  the  Supervisor. 

We  first  model  the  open-loop  hybrid  system  using  CheckMate.  CheckMate  is 
a  verification  tool  for  event-driven  hybrid  systems  for  Matlab/Simulink  environ¬ 
ment,  recently  developed  at  Carnegie  Mellon  University [7].  The  resulting  model 
is  illustrated  in  figure  5. 

The  middle  box  of  figure  5,  named  trains,  corresponds  to  a  Switched  Con¬ 
tinuous  Block  (SCSB)  modeling  the  continuous  subsystem.  The  state  vector  x 
models  the  position  of  the  trains,  the  measured  distances  over  the  track  to  pre¬ 
defined  origins  set  to  D  and  G  respectively.  The  continuous  dynamic  is  defined, 
for  the  input  signal  u,  as  satisfying  x  =  /u(x)  for  x  ^  .  The  input  signal  is 

a  multiplexed  vector  u  with  four  signals,  each  one  assuming  a  positive  integer 
value  associated  to  certain  speed  mode  of  operation.  In  this  4-tuple,  two  signals 
take  values  on  the  modes  of  operation  fast  and  slow,  the  other  two  signals  take 
values  on  the  up  and  down  modes.  The  later,  are  two  artificial  modes  introduced 
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Fig.  5.  CheckMate  model  of  the  open-loop  hybrid  system. 


for  modeling  the  position  variable  in  such  a  manner  that  it  assumes  values  only 
in  certain  range  [0,  max]  without  introducing  jumps  in  the  trajectory.  For  a  given 
position  value  of  maximal  position,  max,  we  associate  to  each  component  of  the 
position  vector  a  positive  (up)  mode  in  the  range  [0,  marr/2]  and  a  negative 
(down)  mode  in  the  range  [max/2,  max].  The  switching  function  which  returns 
the  derivative  of  the  state  vector  for  each  values  of  u  is  specified  in  a  m-file  that 
basically  associates  to  each  values  of  the  4-tuple  an  specific  clock  dynamic,  see 

[7],  given  hy  x  =  ±  ^2]^,  where  vi,V2  G  {vfast.ysiow}  and  Vfast  and  Vsiow 

are  the  possible  trains’  speeds. 

The  seven  boxes  aligned  at  the  right  of  the  SCSB  in  Fig.  5  correspond  to 
Polyhedral  Threshold  Blocks  (PTHBs).  Each  PTHB  represents  a  convex  poly¬ 
hedron  parameterized  by  the  matrix  pair  {C,d).  The  output  of  the  block  is  a 
boolean  signal  that  indicates  whether  the  continuous  state  vector  x  (the  block 
input  signal)  lies  within  the  polyhedron  defined  by  Cx  <  d.  In  this  example, 
each  convex  polyhedron  defines  just  a  line  restriction  associated  to  each  position 
of  the  sensors  over  the  tracks.  As  a  consequence,  the  continuous  space  is  divided 
in  regions.  For  instance,  considering  x  =  [xi  X2]^  the  polyhedron  defined  by 
the  pair  {C,d),  where  C  =  [—1  0]  and  D  =  [—A  0]^,  defines  the  line  constraint 
-^1  <  ~A  or  xi  >  A.  For  this  example  of  PTHB,  the  output  signal  is  true  only 
for  values  x\  >  A.  Observe  that  each  PTHB  of  the  figure  is  labeled  according  to 
this  criterion. 

The  four  boxes  aligned  at  the  left  of  Fig,  5  correspond  to  Finite  State  Machine 
Blocks  (FSMB)  modeling  the  discrete  subsystem.  The  input  events  of  these 
blocks  are  multiplexed  signals  with  the  events  from  the  PTHBs.  The  triggering 
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criterion  adopted  for  all  events  is  rising  edge.  The  two  upper  FSMBs  represent 
the  up /down  logic.  In  particular,  we  model  the  behavior  of  the  system  in  such  a 
way  that  the  occurrence  of  event  D  (G)  forces  the  up  mode,  turns  positive  the 
variable  related  to  the  position  of  train  1  (2).  Also,  the  occurrence  of  event  B  (F) 
forces  the  down  mode,  i.e.,  turns  negative  the  variable  related  to  the  position  of 
train  1  (2).  For  convenience,  locations  B  and  F  are  also  associated  to  the  middle 
of  the  trajectory  of  each  train.  Figure  6  shows  two  FSMBs  representing  up/down 
logic  for  each  train. 
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Fig.  6.  FSMBs  representing  the  up /down  logic  of  each  train. 


In  Figure  7  the  two  FSMBs  represent  a  nondeterministic  logic  for  the 
slow/fast  modes.  Note,  for  instance,  that  when  event  C  (G)  happens  there  is 
a  nondeterministic  choice  of  the  next  mode,  fast  or  slow,  for  train  1  (2).  At  the 
other  hand  the  occurrence  of  event  D  (E)  forces  the  retaking  of  the  fast  mode 
of  train  1  (2).  The  occurrence  of  other  events  leads  to  nondeterministic  choices. 
Note  that  the  representation  of  the  discrete  part  of  the  hybrid  system  through 
four  FSMBs  simplifies  enormously  and  results  in  a  more  intuitive  model. 
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Fig.  7,  FSMBs  representing  the  slow/fast  logic  of  each  train. 


The  model  in  CheckMate  can  be  simulated  according  to  the  rules  of  Matlab^s 
Simulink  environment.  For  simulation  purposes,  we  set  up  the  following  values 
to  the  hybrid  plant  model:  Sensors:  xi  =  20m  (A),  xi  =  40m  (B),  xi  =  10m 
(C),  xi  =  Om  (D);  X2  =  30m  (E),  X2  =  50m  (F)  and  X2  =  Om  (G);  Modes 
fast/slow:  l.Om/s  and  0.2m/s;  and  Initial  Conditions:  Train  1  at  =  Om 
and  Train  2  at  rr2  ==  40m,  in  down  mode.  Note  that  the  values  assigned  to 
the  sensors  are  consistent  with  the  up/down  modes  of  the  position  variable. 
Not  considering  these  artificial  modes  results  in  an  extended  model,  with  sensors 
values  given  by  Xi  =  20m  (A),  xi  —  40m  (B),  xi  =  70m  (C),  xi  =  80{0)m  (D); 
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X2  ~  30m  (E),  X2  =  50m  (F)  and  X2  =  100(0)m  (G).  The  extended  model  is 
useful  for  visualization  of  simulation  results.  For  instance,  Fig.  9  shows  in  the 
extended  model  some  results  for  a  simulation  time  t  =  450s  and  under  the  above 
parameters,  where  the  possibility  of  train  collision  is  clearly  pointed.  Obviously, 
in  this  case  the  order  of  events  causing  nondeterminism  was  defined  so  that  the 
fast  mode  was  forced  to  be  the  preferred  choice. 

The  calculation  of  a  finite  state  approximation  for  the  hybrid  plant  is  ac¬ 
complished  by  running  a  verification  procedure  in  CheckMate,  In  the  case  of  the 
trains  example,  after  two  iterations  of  approximation  and  refinement,  a  finite 
state  machine  with  817  states  and  936  transitions  is  obtained.  The  approxi¬ 
mation  obtained  in  CheckMate  must  be  treated  before  the  application  of  the 
supervisor  synthesis  procedure,  since  it  is  nondeterministic,  non  minimal,  and 
has  spurious  transitions.  A  set  of  basic  functions  of  a  C-f-f  Library  for  manipu¬ 
lation  of  state  machines  called  Grail  [9]  was  extended  and  applied  for  it,  A  finite 
state  deterministic  and  minimal  machine  representing  the  system  of  trains  with 
44  states  and  92  transitions  is  obtained  for  the  example. 

The  specification  of  the  desired  mutual  exclusion  (shown  in  Fig.  8)  simply 
states  that  the  trains  axe  not  allowed  to  enter  the  shared  piece  of  track  at  the 
same  time.  This  specification  is  given  over  the  output  alphabet  V. 


B.F 


Fig.  8.  Specification  of  the  desired  behavior. 


By  application  of  supervisor  synthesis  procedure  described  in  [6],  imple¬ 
mented  also  in  a  Grail  function,  a  supervisor  with  18  states  and  27  transitions 
is  found.  The  resulting  supervisor  represents  the  DES  supervisor  of  Sec.  3,  and 
by  application  of  Alg.  1,  the  C/E  supervisor  can  be  found. 

After  the  succeeding  synthesis  of  the  supervisor  for  the  hybrid  system,  it  is 
possible  to  simulate  the  closed  loop  behavior  in  CheckMate.  The  C /E  supervisor, 
as  defined  in  Sec.  2,  cannot  be  implemented  directly  in  CheckMate  due  to  the 
assumption  of  synchronicity  of  signals  that  is  not  respected  in  the  simulation 
of  two  interconnected  FSMB.  Substitution  of  the  original  discrete  subsystem  by 
the  synthesized  supervisor  would  be  a  valid  simulation  option,  if  the  synthesis 
procedure  had  not  been  based  in  approximations  of  the  plant,  since  approxima¬ 
tions  include  additional  sequences  in  the  original  and  closed-loop  system.  The 
synchronous  composition  of  supervisor  and  discrete  subsystem,  connected  to  the 
continuous  subsystem  is,  in  general,  a  correct  simulation  option,  due  to  the  syn¬ 
chronicity  assumption  of  Sec.  2.  Figure  9  shows  a  simulation  case  for  the  closed 
loop  of  last  case,  putting  in  evidence  the  speed  changes  (fast  to  slow  mode)  of 
train  1  in  order  to  avoid  collisions. 
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Fig.  9.  Sample  simulation  of  the  open-loop  and  closed- loop  system. 


6  Discussion 

This  paper  presents  the  solution  to  a  class  of  supervisory  control  problems  for 
hybrid  systems  with  both  continuous  and  discrete  dynamics.  The  approach  is 
illustrated  with  an  in  example  which,  to  our  knowledge,  is  the  first  published  ex¬ 
ample  of  the  computation  of  a  discrete-state  supervisor  directly  from  a  computer 
model  of  a  hybrid  system. 
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Abstract.  A  necessary  and  sufficient  condition  for  the  reachability  of 
a  piecewise-linear  hybrid  system  is  formulated  in  terms  of  reachability 
of  a  finite-state  discrete-event  system  and  of  a  finite  family  of  affine 
systems  on  a  polyhedral  set.  As  a  subproblem,  the  reachability  of  an 
affine  system  on  a  poly  tope  is  considered,  with  the  control  objective  of 
reaching  a  particular  facet  of  the  polytope.  If  the  polytope  is  a  simplex, 
necessary  and  sufficient  conditions  for  the  solvability  of  this  problem  by 
affine  state  feedback  are  described.  If  the  polytope  is  a  multi-dimensional 
rectangle,  then  a  solution  is  obtained  using  continuous  piecewise-affine 
state  feedback. 

Keywords  and  Phrases:  Piecewise-linear  hybrid  systems,  polyhedral 
set,  simplex,  multi-dimensional  rectangle,  facet,  reachability,  control  law. 


1  Introduction 

The  purpose  of  this  paper  is  to  present  results  on  the  reachability  and  control 
synthesis  of  piecewise-linear  hybrid  systems. 

Many  engineering  systems  can  in  a  first  approximation  be  described  by  a 
piecewise-linear  hybrid  system.  The  computational  and  complexity  issues  of  this 
class  of  systems  seem  comparatively  simple.  Therefore  this  class  merits  attention 
for  the  development  of  control  theory. 

Hybrid  systems  have  been  investigated  since  the  1980’s,  see  [5]  for  references. 
The  class  of  piecewise-linear  hybrid  systems  studied  in  this  paper  may  be  consid¬ 
ered  as  a  subclass  of  the  class  of  piecewise-linear  systems,  introduced  by  Sontag 
in  [14]  (see  also  [16]).  Piecewise-linear  hybrid  systems  are,  in  regard  to  the  geom¬ 
etry  of  the  spaces,  based  on  polyhedral  sets.  The  fact  that  polyhedral  sets  can  be 
described  by  finite-dimensional  parameters,  makes  these  sets  a  suitable  class  of 
objects  for  control  and  system  theory  of  hybrid  systems.  The  class  of  piecewise- 
linear  hybrid  systems  is  therefore  useful  both  because  many  engineering  systems 
can  be  modelled  by  it  and  because  of  its  mathematical  properties. 

*  Research  is  supported  in  part  by  the  Project  Verification  of  Hybrid  Systems  (VHS, 
Esprit  Project  26270)  of  the  European  Commission. 
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Although  control  synthesis  for  hybrid  systems  has  been  investigated  by  many 
researchers  (e.g.  by  S.  Sastry  and  co-workers),  it  is  fair  to  say  that  the  current, 
status  of  control  theory  for  hybrid  systems  is  far  from  satisfactory.  No  review 
of  the  available  literature  will  be  presented  here.  The  reader  is  referred  to  the 
proceedings  of  workshops  [17,12]. 

The  results  of  the  paper  concern  reachability  of  piecewise-linear  hybrid  sys¬ 
tems.  First,  the  approach  of  [19]  is  used  to  show  that  reachability  of  a  hybrid 
system  is  equivalent  to  reachability  of  a  discrete-event  system  and  of  a  family 
of  continuous-space  affine  systems.  The  result  is  based  on  a  natural  decompo¬ 
sition  of  a  hybrid  system  involving  the  concepts  of  arrival  set,  departure  set, 
and  of  a  discrete-event  system  for  the  switches  from  departure  sets  to  arrival 
sets.  Motivated  by  this  result,  the  reachability  problem  for  a  continuous-space 
affine  system  is  formulated  as  whether  there  exists  a  control  law  such  that  the 
closed-loop  system  reaches  from  an  arbitrary  initial  state  a  particular  facet  of  the 
polyhedral  set,  without  reaching  other  facets  first.  Particular  attention  is  payed 
to  the  situations,  where  the  state  set  is  a  simplex  or  a  multi-dimensional  rect¬ 
angle.  On  a  simplex,  the  solvability  of  the  continuous-state  reachability  problem 
using  affine  state  feedback  is  equivalent  to  the  existence  of  a  solution  of  a  set  of 
linear  inequalities  corresponding  to  input  vectors  at  the  vertices  of  the  simplex. 
This  solution  is  treated  in  full  detail  in  a  separate  paper,  see  [7];  it  is  a  nice 
application  of  linear  system  theory  and  of  convex  analysis.  Reachability  of  affine 
systems  on  multi- dimensional  rectangles  can  be  handled  similarly,  provided  that 
continuous  piecewise-affine  state  feedback  is  allowed.  The  proof  is  based  on  the 
fact  that  any  pol>1:ope  admits  a  triangulation  in  terms  of  simplices. 

This  paper  is  organized  as  follows.  Section  2  contains  a  definition  of  a  continu¬ 
ous-time  piecewise-linear  hybrid  system  and  the  formulation  of  the  reachability 
problem.  Concepts  and  a  theorem  on  the  reachability  of  a  hybrid  system  are 
stated  in  Section  3.  In  Section  4  reachability  and  control  of  affine  systems  on 
simplices  and  multi-dimensional  rectangles  is  considered.  Conclusions  are  stated 
in  Section  5. 

2  Problem  Formulation 

In  this  section  a  definition  of  a  piecewise-linear  hybrid  system  is  stated  and  the 
problem  of  reachability  of  such  a  system  is  formulated. 

Throughout  the  paper,  the  notion  of  polyhedral  sets  plays  a  prominent  role. 
Polyhedral  sets  are  subsets  of  IR^,  (iV  e  IN),  described  by  a  finite  number  of 
linear  equalities  and  inequalities.  A  polyhedral  set  that  is  bounded  is  a  polytope, 
and  is  characterized  as  the  convex  hull  of  a  finite  number  of  points:  the  vertices 
of  the  polytope.  A  facet  of  a  polyhedral  set  Pn  C  IR^  is  the  intersection  of 
Pj\i  with  a  supporting  hyperplane,  such  that  the  dimension  of  the  intersection  is 
N  ~1,  For  further  terminology  on  polyhedral  sets  see  [13,15,21]. 

Definition  2.1.  A  (time-invariant  continuous-time)  piecewise-linear  hybrid  sys¬ 
tem  (PLUS)  consists  of  an  automaton  UEcdif),  combination  with  a 

\Q\-tuple  of  affine  systems  {A{q),  B{q),C{q),  D{q),a{q),c(q)),  {q  G  Q),  interact¬ 
ing  in  the  following  way. 
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Given  a  mode  q^Q,  the  continuous  state  Xg  evolves  according  to  the  affine 
differential  equation 

Xg{t)  =  A{q)xg{t)  +  B(q)u{t)  +  a{q),  x,(to)  =  x+ ,  (1) 

y{t)  =  C{q)Xg{t)  +  D{q)u{t)  +  c(q), 

with  Xq  e  Xq  and  u£U.  The  state  set  Xq  and  the  input  set  U  are  assumed  to  be 
polyhedral  sets.  As  soon  as  a  discrete  input  event  e  €  Ein  is  applied,  or  an  event 
e  €  Ecd  (i-e.  an  event  generated  by  the  continuous  dynamics)  occurs,  because  the 
continuous  state  has  reached  the  guard  Gq(e)  C  dXq,  a  discrete  transition  takes 
place  according  to  the  transition  map  f : 

if  x~_  =  (5)  €  Gq-{e)  or  if  e  e  Ein  occurs,  then 

q^  —  f{q 

In  the  new  discrete  mode  q'^ ,  the  evolution  of  the  new  continuous  state  Xq+  is 
described  by  differential  equation  (1),  with  q  replaced  by  q'^ ,  and  with  initial  value 
x'^^  determined  by  the  affine  reset  map 

+br{q~,e,q-'-). 

In  order  to  make  the  system  well  defined,  we  assume  that: 

1.  At  any  fixed  time  only  a  finite  number  of  discrete  transitions  can  occur. 

2.  On  any  finite  interval  only  a  finite  number  of  discrete  transitions  can  occur 
( non-Zenoness). 

In  the  definition  of  a  PLHS  the  input  set  U  and  the  state  sets  Xq  C  IR^^, 
{q  C  Q),  are  polyhedral  sets.  In  Section  4  attention  is  first  restricted  to  state 
sets  that  are  simplices  and  later  on  multi-dimensional  rectangles  are  considered. 

Control  problems  for  hybrid  systems  require  conditions  for  their  solvability.  A 
condition  that  is  required  for  the  solution  of  many  such  problems  is  that  the  sys¬ 
tem  is  reachable,  according  to  the  definition  provided  below.  Before  computing 
a  control  law  one  often  determines  whether  the  considered  system  is  reachable, 
although  in  the  approach  of  the  present  paper,  checking  reachability  and  con¬ 
struction  of  a  control  law  may  be  combined.  However,  verification  of  reachability 
of  a  PLHS  is  theoretically  and  practically  difficult,  because  of  the  extent  of  the 
external  behavior  of  a  PLHS  and  because  of  the  complexity  of  computations  for 
this  clgiss  of  systems. 

Definition  2.2.  Consider  a  PLHS. 

(a)  The  state  {qi,Xq^^i)  6  Q  x  X  is  said  to  be  reachable  from  the  initial  state 
{qo^XqQ^o)  €  Q  X  X  if  there  exist  two  finite  sequences, 

{(ti,  Gi)  6  T  X  Ein\i  =  1, . . . ,  Tn},  {ui  :  ti-i-i)  U\i  =  1, . . . ,  m} , 

such  that  the  PLHS,  starting  at  state  {qo,XqQ^o)  and  with  discrete  and  contin¬ 
uous  input  functions  the  sequences  above,  moves  to  state  {qi,Xq^^i)  G  Q  x  X 
at  time  tm+i,  or,  stated  differently,  at  time  tm+i,q  =  qi  ond  Xq^{tm+i)  = 
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(b)  The  PLHS  is  said  to  be  reachable  from  the  initial  state  (qo.Xg^^o)  eQ  x  X 
if  every  e  Q  x  X  is  reachable  from  the  initial  state. 

Problem  2.1.  Consider  a  PLHS.  Determine  necessary  and  sufficient  conditions 
on  this  system  such  that  it  is  reachable  from  the  initial  state. 

Problem  2.1  is  motivated  by,  for  example,  path  planning  for  robots  and  by 
chemical  process  control.  Because  the  class  of  PLHS  has  a  large  extent,  an  ana¬ 
lytic  solution  is  likely  to  be  intractable.  Thus,  the  reachability  problem  may  be 
undecidable  or,  if  decidable,  of  large  complexity. 

A  major  contribution  to  the  reachability  problem  for  hybrid  systems  was  pre¬ 
sented  by  G.  Lafferriere,  G.  Pappas,  and  S.  Sastry  in  [9,10].  That  approach  uses 
the  notion  of  0-minimality  in  combination  with  the  concept  of  bisimulation.  It 
is  shown  that  the  reachability  problem  is  decidable  if  the  hybrid  system  satisfies 
certain  conditions.  The  approach  of  the  present  paper  differs  from  that  using 
the  O-minimality  approach  in  several  respects:  (1)  The  approach  concerns  only 
piecewise-linear  hybrid  systems.  (2)  The  reset  map  is  an  affine  map.  (3)  A  par¬ 
ticular  decomposition  method  is  used  that  is  not  considered  in  the  O-minimality 
approach.  The  work  is  inspired  by  examples  of  hybrid  systems  that  are  models 
of  engineering  systems. 

The  problem  of  reachability  of  PLHS  or  closely  related  systems  has  also 
been  investigated  by  other  researchers.  O.  Maler,  T.  Dang,  and  co-workers  have 
developed  an  approach  to  approximate  reachability  (see  [1]),  based  on  over  ap¬ 
proximating  the  reachable  set  by  an  orthogonal  polyhedron.  A.  Bemporad,  M. 
Morari,  and  F.  Torrisi  have  developed  a  computational  approach  to  determine 
reachability  of  discrete- time  PLHS  (see  e.g.  [2],  [3]).  For  a  polyhedral  set  of  initial 
states  they  numerically  approximate  the  subset  of  reachable  states  at  subsequent 
times.  This  method  is  effective  for  a  comparatively  small  number  of  time  steps 
and  for  PLHS  with  state  sets  of  relatively  low  dimension. 

3  Reachability  of  Hybrid  Systems 

The  approach  to  the  reachability  problem  discussed  in  this  paper  is  to  decompose 
it  into  a  reachability  problem  for  an  automaton  on  the  one  hand,  and  a  finite 
set  of  reachability  problems  for  continuous-time  polyhedral  linear  systems  on 
the  other.  The  reachability  problem  for  the  automaton  is  easily  solved  by  direct 
computation.  The  resulting  reachability  problem  for  affine  systems  requires  some 
analysis  and  will  be  discussed  in  the  next  section. 

The  approach  to  the  reachability  problem  presented  in  this  section  was  first 
described  by  the  second  author  in  [19]  but,  compared  with  that  reference,  several 
definitions  have  been  sharpened  and  the  theorem  strengthened. 

Definition  3.1.  Consider  a  PLHS. 

(a)  A  departure  set  (or  exit  set)  of  this  system  is  defined  to  be  either  a  guard, 
^q{^)  G  dXq,  Ve  G  Ecd,  Q  ^  Q,  or  a  set  of  the  form 

D{q~,e,q+,A++) 
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X-  e  X,-\q+ =  f{q 


e), 


}■ 


^Ar{q  ,e,q+)x^_  +br{q  ,e,5+)eA++ 

Ve  e  Ein,  A++  C  X,+  a  polyhedral  set,  V9“,5+  g  Q,  q~  q'^ ■ 
(b)  An  arrival  set  (or  entry  setj  is  defined  to  be  a  set  of  the  form 
AR{q~,e, 


G  Xq+  \3x^_  £  D  C  Xq-  such  that, 
=  f{q'',x-,e),  =  Ar(q- ,e,q+)x 


+  br{q  .e.?'*')}’ 


Vg”,g+  €  Q,  q~  Ve  g  EinUEcd,  VD  C  X,-  a  polyhedral  set. 

A  departure  set  in  Xg  may  be  interpreted  as  a  set  from  which  the  state 
trajectory  may  leave  the  state  set  Xq.  Such  a  departure  takes  place  either  at 
a  guard  by  a  continuous- dynamics  event  e  £  Ecd  or  from  a  departure  set  by 
application  of  an  input  event  e  G  Ein.  An  arrival  set  is  a  subset  in  which  the 
state  trajectory  will  enter  the  state  set  Xq  directly  after  a  transition. 

Definition  3.2.  Consider  a  PLUS.  The  controllability  set  for  {q,  Xq^i)  £QxX 
is  defined  to  be  the  set  of  all  states  Xg^o  £  Xg  from  which  Xq^i  can  be  reached 
without  leaving  Xgi 

Conset((g,Xg,i)) 

_  f  e  Q  X  Xq\3to,ti  G  T,  to  <  ti,3u  :  [to,ti)  U,  such  that,  1 

“  ^g,0?  Xq(ti^  ~  ^9,1’  ^  (^0?ti),  Xqitf  G  Xq  J 

The  controllability  set  of  a  subset  Sq  C  Xq  is  defined  to  be  the  set 

Conset{{q,  Sg))  =  Ua:^^es^Conset{{q,Xg^i)) 

Definition  3.3.  Consider  a  PLUS.  Assume  there  exists  a  finite  collection  of 
disjoint  sets  of  the  form 

A  =  {A{q,  k)  ^  Xg\k  £  {1, . . .  jngj ,  q  £  Q}. 

Assume  further  that  for  every  arrival  set  ARiq" ,e,q'^ ,Xg-),  for  q~,q'^  £  Q, 
and  e  £  Ein  U  Ecd,  there  exists  a  subset  of  the  A-sets  such  that 

AR{q~,e,q''',Xg-)  C  fc).  (2) 

The  collection  is  called  a  collection  of  A-sets.  Define  the  corresponding  A-auto- 
maton  as  a  possibly  non-deterministic  finite  automaton  {A,  Ein  U  Ecd,  fA,Ao), 
with  initial  state  Aq  £  A,  and  partial  function  fA'.  Ax  {Ein  U  Ecd)  ->  A,  defined 
by 

(  A{q+,m),  if 

either  A{q~ ,  k)  C  Conset (D {q~  ,e,q~^ ,A{q'^ ,m))), 
or  A{q~,k)  C  Conset(Gg-  (e))  A  (3) 

Ar{q~,e,q+)x~_  +  br{q~,e,q+)  £  A{q^,m), 
for  all  x~^  £  Gg-{e). 


fA{A{q  ,k),e)^{ 
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In  the  case  of  a  piecewise-linear  hybrid  system,  boundaries  of  departure  sets 
and  guards  are  assumed  to  be  facets  of  the  relevant  polyhedral  state  set.  Oth¬ 
erwise  the  polyhedral  state  set  must  be  further  decomposed  into  smaller  poly¬ 
hedral  sets  such  that  boundaries  of  departure  sets  and  guards  are  facets.  The 
question  is  thus:  how  to  determine  the  controllability  set  of  a  facet  of  a  polyhe¬ 
dral  set?  In  Section  4  this  problem  is  studied  for  state  sets  that  are  simplices  or 
multi-dimensional  rectangles.  A  condition  is  formulated  that  is  equivalent  to  the 
controllability  set  of  a  facet  being  the  full  state  set.  In  this  situation,  condition 
(3)  in  Definition  3.3  of  the  A- automaton  is  always  satisfied.  This  implies  that  an 
A- automaton  exists,  but  its  state  set  may  be  infinite.  Definition  3.3  imposes  a 
restriction  because  the  state  set  is  assumed  to  be  finite.  This  is  discussed  again 
below. 

Theorem  3.1.  Consider  a  PLHS.  Assume  that  there  exists  a  finite  collection 
of  sets  of  the  form 

A  =  {A(q',  A:)  C  G  {1, . . . ,  n^},  ^  G  Q}, 

such  that  the  conditions  of  Definition  3.3  hold.  Then  the  PLHS  is  reachable  from 
any  initial  state  if  and  only  if: 

1.  The  A~automaton  is  reachable  (every  state  Ai  G  A  is  reachable  from  any 
initial  state  Aq  G  A ); 

2.  for  any  e  QxX  there  exists  a  set  A{qi,k)  G  A  such  that  (qi.Xg^^i) 

is  reachable  from  all  initial  states  in  A(qfi,A:)  without  leaving  equiva¬ 
lently,  for  any  {qi,Xg^^i)  e  Q  x  X  there  exists  a  set  A{qi,k)  G  A  such  that 
A(gi,A;)  C  Conset((gi,a:g,,i)). 

Theorem  3.1  is  a  strengthened  version  of  [19,  Th.  8].  It  provides  a  necessary 
and  sufficient  condition  for  reachability  of  a  piecewise-linear  hybrid  system  in 
terms  of  conditions  and  calculations.  The  main  assumption  for  this  result  is 
the  existence  of  a  finite  collection  of  A-sets.  It  is  related  to  the  concept  of  O- 
minimality  and  the  approach  to  reachability  developed  by  G.  Lafferriere,  G. 
Pappas,  and  S.  Sastry,  see  [9,10].  The  first  condition  for  reachability  of  the  PLHS, 
reachability  of  the  A-automaton,  is  simple  to  check  by  a  computer  program. 
The  second  condition  is  a  new  problem.  The  problem  is  whether  there  exists 
an  input  that  transfers  the  system  from  an  A-set  to  a  final  state.  This  problem 
has  been  investigated  in  control  theory  in  some  generality  but  not  for  piecewise- 
linear  systems  on  polyhedral  sets  as  far  as  the  authors  know.  An  approach  to 
this  problem  is  to  construct  a  Lyapunov  function  that  assures  convergence  to 
a  point.  For  an  affine  system  on  a  simplex,  further  inspiration  for  tackling  this 
problem  may  be  taken  from  the  discussion  in  Section  4. 

Example  3.1.  Control  of  a  conveyor  belt  system.  This  example  has  been  de¬ 
scribed  in  detail  in  [18].  There,  a  conveyor  belt  system  was  modelled  as  a 
piecewise-linear  hybrid  system.  The  arrival  sets  are  simple  to  formulate  because 
they  correspond  mostly  to  the  arrival  of  a  tray  with  manufacturing  parts  at  either 
the  front  end  of  the  belt  or  at  the  mid-point  of  the  belt.  The  A-automaton  based 
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on  arrival  sets  as  A-sets  has  about  26  states  and  84  transitions,  and  was  checked 
to  be  reachable,  using  the  computer  program  UMDES,  developed  by  S.  Lafor- 
tune  at  the  University  of  Michigan.  Reachability  of  any  of  the  continuous-time 
systems  is  also  easily  verified:  the  guards  can  be  reached  by  switching  on  the 
motor  that  drives  the  belt  and  the  departure  sets  equal  the  full  state  set. 

A  first  approach  for  constructing  the  A-sets  is  to  take  them  equal  to  the  full 
arrival  sets, 

A(g,r)  =  AR{q~,e,q,Xg-),  for  r  G  IN  an  index,  and  q~,qeQ,  e  G  E. 

In  some  examples,  like  Example  3.1,  this  choice  suffices,  but  in  general  it  is  too 
restrictive.  Recall  from  Definition  3.3  that  a  transition  between  A-sets  is  not 
defined  if  an  arrival  set  is  not  fully  contained  in  a  corresponding  controllability 
set.  Hence  it  seems  useful  to  split  an  arrival  set  into  two  or  more  subsets  and  to 
take  these  subsets  as  new  A-sets.  This  approach  is  formulated  in  Algorithm  3.1 
below.  The  construction  of  A-sets  is  similar  to  the  bisimulation  algorithm,  see 
[9] ,  except  that  it  refers  to  the  splitting  of  arrival  sets  only. 

Algorithm  3.1.  Consider  a  PLUS  and  the  collection  of  arrival  sets 

(  AR{q-,e,q+,D)  c  Xg+\q-,q+  G  Q,  g" 

AR  =  <  and  either  {e  G  Ein  and  D  =  D{q~~,e,  g"^,  Xg+)} 
or  {e  G  Ecd  D  —  Gq-  (e)} 

Consider  further  a  terminal  state  {qf^^gfj)  €  Q  x  A  to  be  reached. 

1.  Initialization  set 

^0 (g/ ,  A;)  -  AR{q- ,  e,  g/ ,  D)  n  Conset ((g/ ,Xq^j)), 
for  q~  eQ,  e  G  Ein  U  Ecd, 

D  =  Gq-{e)  or  D  =  D{q^,e,qf,Xqf), 

Aoiq^k)  =  Ai?(g",e,g,D), 

e  e  EinU  Ecd, 

D-=^Gq-{e)  or  D  =  D(g“,e,g,Xg). 

2.  Backward  recursion.  Construct  the  sets, 

Bk^i{q,m)  =  Afc(g“,mi)  nConset(D(g",e,g,  Afc(g,m2))), 
g,  g  G  Q,  e  G  Ein  U  Ecd,  m,  mi, m2  G  IN. 

Then  produce  a  disjoint  collection  of  the  collection  of  sets  and  denote 
these  by 

Ak+i  =  {AA:+i(g,m)  C  Xq\m  G  {1, . . .  ,nA:+i},g  G  Q}. 

Define  the  map, 

h  :  Pwrset{Pwrset{Q  x  A))  -)•  Pwrset{Pwrset{Q  x  A)),  Ak+i  —  h{Ak). 
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Two  questions  for  the  above  algorithm  require  further  study: 

1.  Does  the  algorithm  terminate  in  a  finite  number  of  steps? 

2.  If  the  algorithm  terminates  (thus  Ak  =  HAk)  for  some  K  €  IN),  is  the 
resulting  collection  Ak  finite,  i.e.  does  there  exist  tua  €  IN  such  that 

Ak  =  {AK{q,  m)  C  Xq\m  G  {1, . . . ,  q  G  Q}? 

If  both  questions  have  been  answered  affirmatively.  Theorem  3.1  provides  an 
equivalent  condition  for  reachability.  Note  however  that  for  the  construction  of 
A-sets,  the  computation  of  controllability  sets  is  required.  In  the  next  section 
we  will  consider  this  issue,  in  case  the  set  to  be  reached  is  a  facet  of  the  state 
set.  Often  the  state  set  can  be  partitioned  in  such  a  way  that  reachability  of  a 
polyhedral  subset  can  be  reformulated  as  reachability  of  a  facet.  In  these  situa¬ 
tions  the  discussion  of  Section  4  facilitates  the  construction  of  A-sets.  As  soon 
as  Algorithm  3.1  terminates,  and  the  required  collection  of  A-sets  is  obtained, 
the  reachability  of  a  piecewise-linear  hybrid  system  may  be  verified  by  checking 
(1)  the  reachability  of  the  A-automaton  and  (2)  the  reachability  of  a  finite  set 
of  systems. 


4  Reaching  Departure  Sets  Using  Feedback  Control 

In  this  section  we  focus  our  attention  on  one  particular  discrete  mode  of  a 
piecewise-linear  hybrid  system,  and  study  the  continuous  dynamics  at  that  spe¬ 
cific  mode.  There  the  continuous  evolution  of  the  system  is  described  by  an  affine 
differential  equation 

x{t)  =  Ax{t)  -b  Bu{t)  +  a,  (4) 

with  A  e  B  e  and  a  G  IR^.  We  assume  that  the  state  set  X  is  a 

(full-dimensional)  convex  polytope  in  IR^.  Also  the  choice  of  inputs  u  G  IR”^ 
is  restricted  to  a  polyhedral  set  U. 

As  soon  as  the  state  x  crosses  one  of  the  facets  of  .P/v,  a  discrete-event  occurs, 
transferring  the  system  to  a  different  mode  with  different  continuous  dynamics. 
So  the  facets  of  completely  consist  of  departure  sets.  We  will  assume  that 
every  facet  of  P^  consists  of  exactly  one  departure  set,  meaning  that  the  discrete 
transition  to  another  mode  only  depends  on  the  facet  of  P^  through  which 
the  state  x  leaves  the  polytope  P^.  So,  to  steer  the  overall  hybrid  system  to 
a  particular  state,  using  the  approach  of  Section  3,  we  first  have  to  answer 
the  question  whether  it  is  possible  to  steer  the  affine  system  (4)  to  a  specific 
facet/departure  set.  Preferably,  this  steering  should  be  implemented  by  a  static 
state  feedback.  So,  in  this  section  we  will  study  the  following  problem: 

Problem  4.1.  Consider  the  system  (4)  with  x  G  Pjv,  and  let  Fi  be  a  facet  of  P^, 
with  normal  vector  m,  pointing  out  of  P^.  For  any  initial  state  xq  e  Pn,  find 
a  time-instant  To  >  0  and  an  input  function  u  :  [0,  To)  — U,  such  that  at  time 
To  the  state  x  leaves  P^  through  the  facet  Pi,  i.e. 
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(i)  \fte[0,To]:  x{t)ePN. 

(ii)  x{To)  £  Fi,  and  To  is  the  smallest  time-instant  in  the  interval  [0,To]  for 
which  x{t)  £  Fi, 

(Hi)  nJx{To)  >  0,  i.e.  the  velocity  vector  x(To)  at  the  point  a:(To)  G  Fi  has  a 
positive  component  in  the  direction  of  ni. 

Furthermore,  this  input  function  u  should  be  realized  by  the  application  of  a 
continuous  feedback  law  u  =  f{x),  with  /  independent  of  the  initial  state  xq. 

Problem  4.1  is  related  to,  but  different  from  the  existence  of  a  control  law 
for  an  affine  system,  such  that  the  closed-loop  system  is  invariant  on  a  polytope, 
see  [4,6,20]. 

Since  the  class  of  all  continuous  feedback  laws  is  very  large,  we  will  focus 
on  solutions  of  Problem  4.1,  using  affine  feedback  (if  Pn  is  a  simplex)  or  con¬ 
tinuous  piecewise-affine  feedback  (if  is  a  multi- dimensional  rectangle).  This 
restriction  enables  us  to  construct  feedback  solutions,  by  making  extensive  use 
of  the  convexity  of  the  problem.  First  however,  we  formulate  a  set  of  necessary 
conditions  for  the  solvability  of  Control  Problem  4,1. 

Proposition  4.1.  Let  P/v  be  a  full- dimensional  polytope  in  IR^  with  ver¬ 
tices  {M  >  N  -\-  1).  Let  Pi,...,Pl  denote  the  facets  of  Pn,  with 

normal  vectors  ni,..,,n£,,  respectively,  pointing  out  of  the  polytope  Pn-  For 
i  G  {1,...,!/},  let  Vi  C  M}  be  the  index  set  such  that  {vj  \  j  £  Vi} 

is  the  set  of  vertices  of  the  facet  Fi.  Conversely,  for  every  j  £  M],  the 

set  Wj  C  {1, . . .  ,T}  contains  the  indices  of  all  facets  of  which  Vj  is  a  vertex. 
Assume  that  Fi  is  the  exit  facet  of  Pn-  If  Control  Problem  f.l  is  solvable  by  a 
continuous  state  feedback  f,  i.e.  if  irrespective  of  the  initial  state  xq  £  Pn,  the 
closed-loop  system 

X  =  Ax  A- Bf{x)  V  a,  x{0)  =  xq, 

has  a  solution  x,  satisfying  conditions  (i) — (Hi)  of  Problem  4-1,  then  there  exist 
inputs  ui, . . . , um  ^  bJ  such  that 

(1)  Vj  £  Vi:  nf  (Avj  H-  Buj  +  a)  >  0, 

(2)  Vz  €  {2, . . . ,  L}  Vj  €  Vi:  nj (Au.  +  Buj  -f-  a)  <  0, 

(3)  Vj  G  {1, . . . ,  M}\Vi  G  Wj:  ri[ (Avj  +  Buj  4-  a)  <  0. 

Proof  Suppose  that  the  continuous  feedback  f  i  Pn  — solves  Problem  4.1. 
Then  the  inputs  Uj  —  f{vj),  (j  =  1, . . . ,  M)  satisfy  (1),  (2),  and  (3). 

Indeed,  at  the  exit  facet  Pi  the  velocity  vector  field  of  the  closed-loop  system 
has  a  positive  component  in  the  ni-direction,  hence  (1)  holds.  Furthermore,  the 
state  of  the  closed-loop  system  cannot  leave  Pjv  through  any  of  the  other  facets, 
irrespective  of  the  initial  state  xq.  This  implies  that  on  these  facets  the  velocity 
vector  field  of  the  closed-loop  system  has  to  point  into  the  polytope  Pn-  This 
condition  remains  valid  at  the  vertices  of  a  facet,  and  therefore  (2)  holds.  Finally, 
the  state  of  the  closed-loop  system  should  reach  the  exit  facet  Pi  in  finite  time. 
If  (3)  would  not  hold,  then  there  would  exist  a  vertex  v  of  Pn,  not  belonging  to 
Pi,  such  that  the  velocity  vector  field  of  the  closed-loop  system  in  v  is  equal  to 
0.  Hence  x{t)  =  v  would  be  a  solution  of  the  closed-loop  system,  never  reaching 
the  exit-facet  Pi.  This  leads  to  a  contradiction.  □ 
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Our  next  goal  is  to  derive  sufficient  conditions  for  the  solvability  of  Problem 

4.1,  For  this  purpose,  we  first  consider  the  case  where  the  convex  polytope  Pj\i 
is  a  simplex  i.e.  a  full-dimensional  polytope  in  IR^,  with  exactly  N  -\- I 
vertices.  In  this  situation,  the  necessary  conditions  of  Proposition  4.1  turn  out 
to  be  sufficient  for  the  construction  of  an  affine  feedback  solution  to  Problem 

4.1. 

Lemma  4.1.  Let  he  a  full- dimensional  simplex  in  IR^  with  affinely  inde¬ 
pendent  vertices  Vi,.,  and  let 

N+l 

Tn+1  :=  {(Ai, . .  .,Xn+i)  e  [0, 1]^+^  |  ^  =  1}. 

For  every  x  £  Sn  there  exists  a  unique  (Ai, . . . ,  Aat+i)  G  such  that  x  = 

Z)j=i  Moreover,  the  corresponding  mapping  ip  :  ^  Tjv+i  is  affine, 

and  thus  continuous. 

Proposition  4.2.  Consider  the  dynamical  system  x{t)  =  Ax{t)  -f  Bu{t)  -f  a, 
with  X  £  and  u  £  U,  and  assume  that  there  exist  inputs  wi, . . .  ,Ujv+i  G  U, 
such  that  at  the  vertices  Vi, . . .  of  the  simplex  Sjsj,  conditions  (1 ) — (3)  of 

Proposition  4.I  are  satisfied.  Define  the  affine  mapping 

N+l 

Ip  :  Tiv+i  — >  U  :  V^(Ai, . . . ,  X^+i)  =  ^  XjUj. 

j=i 

Then  the  mapping  f  :  S'jv  — >  U ,  defined  by  f  ~  ip  o  (p^  is  an  affine  feedback 
solution  of  Control  Problem  4-1. 

Proof.  For  notational  convenience  we  assume  that  the  vertices  . . . ,  ?;iv+i  of 
Sj^  are  numbered  in  such  a  way,  that  for  i  =  1 , . . . ,  TV  -f- 1 ,  vi  is  the  only  vertex 
of  Sn,  not  belonging  to  the  facet  Fj. 

First  we  prove  that  the  state  x  of  the  closed-loop  system  x  =  Ax-\-  Bf{x)-\-a 
cannot  leave  the  simplex  through  any  of  the  facets  F2, . .  .,Fn+i.  Let  i  G 
{2,  ..,,A^  +  1}?  and  consider  the  facet  Fi  with  normal  vector  n^.  Let  p  £  Fi 
and  (Ai, . . . ,  A;v+i)  =  ^{p),  with  Aj  =  0.  Then  condition  (2)  of  Proposition  4.1 
guarantees  that 

nf±\p  =  nf  (Ap  +  Bf(p)  +  a) 

(N+l  N+l  N+l 

A  ^  Ajvj  +  B  Ajuj+  ^  Aja 

7V+1 

=  ^  Xjuf  (Avj  -f-  Buj  A  a)  <0. 

Hence,  on  every  facet  F2, . . . ,  Fn+u  the  velocity  vector  field  of  the  closed-loop 
system  is  pointing  into  the  simplex  Sn,  so  the  state  x  cannot  escape  from  Sn 
through  any  of  these  facets. 
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Next  we  show  that  the  exit  facet  Fi  is  reached  within  finite  time.  For  this 
proof  we  need  the  fact  that  the  normal  vector  ni  can  be  written  as  a  negative 
linear  combination  of  n2, . . .  (see  e.g.  [7,  Lemma  A.3]).  Let  p  €  Sjsj  with 

(^(p)  =  (Ai, . . . ,  Aat+i).  Then  conditions  (1),  (2),  and  (3)  of  Proposition  4.1  and 
an  argument  imply  that 

(jV+l  N+l  N+1 

i=l  J=1 

N+l 

=  Ain^ {Avi  +  Bui  +  a)  +  Aj-n^ {Avj  +  Buj  +  a)  >  0. 

J=2 

Moreover,  the  simplex  is  compact,  so  minjnfa:  \p\  p  e  Sj^}  exists  and  is 
positive.  Therefore  the  state  x  reaches  the  exit  facet  Fi  in  finite  time.  □ 


Remark  4- F  Given  inputs  ui, . . . ,  ujv+i  ^  U  et  the  vertices  ,  t’jv+i  of  th® 

simplex  S'jv,  satisfying  conditions  (1) — (3)  of  Proposition  4.1,  an  affine  feedback 
law  u  =  Fx  g  with  F  €  and  g  G  IR"”  that  solves  Control  Problem 

4.1,  can  be  computed  directly  by  solving  the  linear  equations  uj  —  Fvj  +  g, 
(j  =  1, . . . ,  iV  +  1),  for  F  and  g. 

Example  4.1.  Control  to  a  facet.  Let  ^2  be  the  triangle  in  IR^,  with  vertices 
Vi  =  (-1,0)^,  V2  =  (1, 1)^,  and  V3  =  (1,  -1)^,  and  consider  the  affine  system 

with  state  x  e  S2  and  scalar  input  -1  <  w  <  1.  Consider  Control  Problem  4.1, 
with  Fi,  the  facet  between  V2  and  U3,  as  exit-facet.  This  problem  is  solvable  if 
and  only  if  there  exist  inputs  tti,  1/2,  ns  at  the  vertices  ^^2?  '*^3,  respectively,  such 
that  condition  (1),  (2),  and  (3)  of  Proposition  4.1  are  satisfied.  In  this  example 
these  inequalities  become  ^  <  ui  <1,  <  U2  <1,  and  -1  <  us  <  Upon 

choosing  ui  —  U2  =  0,  and  U3  =  -|,  an  affine  feedback  solution  of  Problem 
4.1  is  given  by 

The  idea  of  the  proof  of  Proposition  4.2  can  be  extended  to  multi- dimensional 
rectangles,  if  we  also  allow  continuous  piecewise-affine  functions  /  :  Pjsf  — >  U 
as  possible  feedback  solutions. 

Proposition  4.3.  Let  Pn  be  a  full- dimensional  convex  polytope  in  IR^  with 
vertices  ui, . . .  (M  >  N  +  1),  and  define 

M 

Tm  ■■=  {(Ai, . . . ,  Am)  e  [0, 1]"  I  ^  A,-  =  1}. 

j  =  l 
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Then  there  exists  a  continuous  and  piecewise- affine  mapping  (p  :  Pn  — >  Tm 
such  that  for  all  x  e  Pn: 

M 

x  =  (5) 

j=i 

The  fact  that  any  point  x  e  Pn  may  be  written  as  a  convex  combination  of 
the  vertices  of  is  obvious.  Note  however  that  such  a  convex  combination  is 
in  general  not  unique,  unless  M  =  N  1.  If  M  >N  +  1,  one  may  construct 
(p  :  Pj^  — >  Tm,  satisfying  (5),  by  subdividing  P/v  in  simplices  (triangulation,  see 
e.g.  [11]).  Then  every  x  e  Pn  is  uniquely  represented  as  a  convex  combination  of 
those  vertices,  that  are  vertices  of  all  the  simplices  of  which  x  is  an  element.  This 
representation  yields  a  continuous  and  piecewise-affine  mapping  cp^  satisfying  (5). 

Proposition  4.4.  Let  Rn  be  the  multi- dimensional  rectangle  defined  by 

Rn  :=  {x  \  \/i  =  l,...,N  :  ai  <Xi<bi}, 

and  consider  the  dynamical  system  x{t)  =  Ax{t)  +  Bu{t)  +  a,  with  x  e  Rn 
and  u  £  U.  Let  Fi  Rn  Pi  {o:  €  IR^  \  xi  =  bi}  be  the  exit  facet  of  Rn  with 
normal  vector  e\.  The  normal  vectors  on  the  other  facets  are  —ei  and  (i  = 
2, . . . ,  N),  Denote  M  =  2^,  and  let  p  :  Rn  ->  Tm  be  a  continuous  and  piecewise- 
affine  mapping,  satisfying  (5),  Assume  that  there  exist  inputs  ^  U, 

such  that  at  the  vertices  of  Rn,  conditions  (l)—(3)  of  Proposition 

4.1  are  satisfied,  and  additionally 

(4)  Vj  =  1, . . . ,  M  :  ej {Avj  +  Buj  +  a)  >  0. 

Define  the  affine  mapping 


M 

:  Tm  — >  U  :  V^(Ai, . . . ,  Am)  =  AjUj. 

j=i 

Then  the  mapping  f  :  Rn  — >  U ,  defined  by  f  =  'ip  o  p,  is  a  continuous  and 
piecewise-affine  feedback  law,  solving  Control  Problem  4-L 

The  proof  of  Proposition  4.4  is  analogous  to  the  proof  of  Proposition  4.2. 
Condition  (4)  is  required  to  guarantee  that  the  state  of  the  closed-loop  system 
reaches  the  exit-facet  Pi  in  finite  time.  However,  for  multi-dimensional  rectangles 
condition  (4)  is  almost  implied  by  conditions  (1)  and  (2);  at  the  vertices  of  the 
exit  facet  conditions  (1)  and  (4)  are  the  same,  and  at  the  other  vertices  (the 
vertices  of  the  facet  G  IR  |  2:1  =  ai}  with  normal  vector  — ei),  condition 

(2)  states  that  e'i{Avj  +  Buj  +  a)  >  0.  The  only  difference  with  condition  (4) 
is  a  >-sign  instead  of  a  >-sign.  So  the  necessary  and  sufficient  conditions  of 
Propositions  4.1  and  4.4  are  almost  equivalent.  Furthermore,  in  Proposition  4.4 
condition  (3)  may  be  omitted,  because  it  is  implied  by  (4). 

Remark  4.2.  The  design  method  of  Proposition  4.4  for  the  construction  of  a 
continuous  piecewise-aflane  feedback  solving  Problem  4.1  is  applicable  to  arbi¬ 
trary  full- dimensional  convex  polytopes  Pjv-  However,  if  Pn  is  not  a  simplex  or 
a  multi-dimensional  rectangle,  the  sufficient  condition  (4)  becomes  restrictive. 


Control  of  Piecewise-Linear  Hybrid  Systems  on  Simplices  and  Rectangles  273 


Propositions  4,1,  4.2,  and  4.4  yield  necessary  and  sufficient  conditions  for 
the  solution  of  Control  Problem  4.1  for  simplices  and  rectangles.  So,  if  for  a 
hybrid  system  the  state-set  at  each  discrete  mode  belongs  to  this  class,  the 
question  of  reachability  of  a  departure  set  may  be  translated  into  the  solvability 
of  a  system  of  linear  inequalities.  Therefore,  the  verification  may  be  carried  out, 
using  existing  software  packages,  e.g.  [8]. 

5  Conclusions 

The  contribution  of  this  paper  to  control  of  hybrid  systems  concerns  reachability 
and  control  law  synthesis.  First,  an  equivalent  condition  for  reachability  of  a 
piecewise-linear  hybrid  system  was  formulated  in  terms  of  reachability  of  a  finite- 
state  discrete-event  system  and  of  a  finite  family  of  affine  systems  on  a  polyhedral 
set.  Next,  an  equivalent  condition  for  reachability  of  an  affine  system  on  a  simplex 
was  derived,  for  the  control  objective  of  reaching  a  particular  facet  of  the  simplex. 
This  result  was  extended  to  multi- dimensional  rectangles.  The  solution  is  based 
on  the  construction  of  a  continuous  (piecewise)  affine  control  law. 

Further  research  is  required  into  the  reachability  of  an  affine  system  on  a 
general  polytope.  Computational  aspects  of  the  construction  of  a  control  law 
should  be  studied.  For  this,  triangulation  of  polyhedral  sets  is  needed,  involving 
concepts  of  discrete  and  computational  geometry.  Symbolic  computation  seems 
well  suited  for  this  operation.  Application  of  the  results  to  engineering  systems 
also  requires  attention. 
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Abstract.  The  assume-guarantee  paradigm  is  a  powerful  divide- and- 
conquer  mechanism  for  decomposing  a  verification  task  about  a  system 
into  subtasks  about  the  individual  components  of  the  system.  The  key 
to  assume-guarantee  reasoning  is  to  consider  each  component  not  in  iso¬ 
lation,  but  in  conjunction  with  assumptions  about  the  context  of  the 
component.  Assume-guarantee  principles  are  known  for  purely  concur¬ 
rent  contexts,  which  constrain  the  input  data  of  a  component,  as  well  as 
for  purely  sequential  contexts,  which  constrain  the  entry  configurations  of 
a  component.  We  present  a  model  for  hierarchical  system  design  which 
permits  the  arbitrary  nesting  of  parallel  as  well  as  serial  composition, 
and  which  supports  an  assume-guarantee  principle  for  mixed  parallel- 
serial  contexts.  Our  model  also  supports  both  discrete  and  continuous 
processes,  and  is  therefore  well-suited  for  the  modeling  and  analysis  of 
embedded  software  systems  which  interact  with  real-world  environments. 
Using  an  example  of  two  cooperating  robots,  we  show  refinement  between 
a  high-level  model  which  specifies  continuous  timing  constraints  and  an 
implementation  which  relies  on  discrete  sampling. 


1  Introduction 

In  the  automatic  verification  of  systems  with  very  large  state  spaces,  the  model¬ 
checking  task  needs  to  be  decomposed  into  subtasks  of  manageable  complexity. 
It  is  natural  to  decompose  the  verification  task  following  the  component  struc¬ 
ture  of  the  design.  However,  an  individual  component  often  does  not  satisfy  its 
requirements  unless  the  component  is  put  into  the  right  context.  Thus,  in  or¬ 
der  to  verify  each  component  individually,  we  need  to  make  assumptions  about 
its  context,  namely,  about  the  other  components  of  the  design.  This  reaisoning 
is  circular:  component  A  is  verified  under  the  assumption  that  context  B  be¬ 
haves  correctly,  and  symmetrically,  B  is  verified  assuming  the  correctness  of  A. 
The  assume-guarantee  paradigm  provides  a  systematic  theory  and  methodology 
for  ensuring  the  soundness  of  the  circular  style  of  postulating  and  discharging 
assumptions  in  component- based  reasoning. 

*  Support  for  this  research  was  provided  in  part  by  the  AFOSR  MURI  grant  F49620- 
00-1-0327,  and  the  DARPA  SEC  grant  F33615-C-98-3614,  the  MARCO  GSRC  grant 
98-DT-660,  the  NSF  ITR  grant  CCR-0085949. 
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When  components  are  composed  in  parallel,  context  assumptions  constrain 
the  inputs  to  a  component.  Assume-guarantee  principles  for  parallel  composition 
are  advocated,  among  others,  by  [MC81,AL95,McM97,AH99],  and  by  [TAKB96, 
AH97]  in  a  real-time  setting.  If  components  are  composed  in  series,  context 
assumptions  constrain  the  entry  configurations  of  a  component.  An  assume- 
guarantee  principle  for  serial  composition  is  given  in  [AGOO].  In  hierarchical 
design,  it  is  often  useful  to  nest  parallel  and  serial  composition.  This  is  espe¬ 
cially  true  for  embedded  software,  where  serial  composition  occurs  at  multiple 
levels  of  granul^Lrity  (e.g.,  software  procedures;  modes  of  operation;  exception 
handling),  and  so  does  parallel  composition  (e.g.,  hardware  modules;  software 
threads;  environment  interaction).  We  provide  an  assume-guarantee  principle 
for  the  case  where  a  context  can  contain  both  parallel  and  serial  components, 
arbitrarily  nested. 

For  this  purpose,  we  use  a  formal  model  which  is  called  Masaccio,  in  honor  of 
the  Italian  fresco  painter  who  is  credited  with  inventing  perspective.  The  Masac¬ 
cio  language  was  defined  in  [HenOO];  we  modify  it  slightly  in  order  to  obtain  a 
general  assume-guarantee  principle.  Masaccio  is  a  formal  model  for  hybrid  dy¬ 
namical  systems  which  are  built  from  atomic  discrete  components  (difference 
equations)  and  atomic  continuous  components  (differential  equations)  by  paral¬ 
lel  and  serial  composition,  arbitrarily  nested.  Data  is  represented  by  variables; 
control  by  locations.  The  syntax  of  components  includes  six  operations:  besides 
parallel  and  sequential  composition,  data  connections  are  built  by  variable  re¬ 
naming,  control  connections  by  location  renaming,  data  abstractions  by  variable 
hiding,  and  control  abstractions  by  location  hiding.  The  formal  semantics  of  each 
component  consists  of  an  interface,  which  determines  the  possible  ways  of  using 
the  component,  and  a  set  of  executions,  which  define  the  possible  behaviors  of 
the  component  in  real  time.  The  intended  use  of  Masaccio  is  to  provide  a  for¬ 
mal,  structured  model  for  software  and  hardware  that  interacts  with  a  physical 
environment  in  real  time.  Parallel  composition  is  conjunctive:  it  typically  com¬ 
bines  actors  (software  threads,  sensors,  actuators,  etc.);  serial  composition  is 
disjunctive:  it  typically  combines  modes  of  operation  (time-triggered  and  event- 
triggered  mode  switching,  degraded  and  fault  modes,  etc.).  Masaccio  conserva^ 
tively  extends  Reactive  Modules  [AH99,AH97],  which  provide  parallel  but  no 
serial  composition,  and  it  inherits  the  mixing  of  discrete  and  continuous  behav¬ 
ior  firom  Hybrid  Automata  [ACH'‘'95,Hen96],  which  are  not  hierarchical.  The 
parallel  composition  of  Masaccio  is  synchronous;  asynchronicity  can  be  modeled 
as  in  [AH99). 

We  demonstrate  that  Masaccio  supports  hierarchical,  component-based  de¬ 
sign  and  analysis.  In  particular,  we  prove  the  soundness  of  (noncircular)  compo¬ 
sitional  proof  rules  for  both  parallel  and  serial  composition,  and  the  soundness 
of  a  (circular)  assume-guarantee  proof  rule,  which  permits  assumptions  about 
mixed  parallel-serial  contexts.  Several  key  insights  are  necessary  to  enable  the 
assume-guarantee  principle.  First,  assume-guarantee  reasoning  is  sound  only  for 
components  that  cannot  deadlock  internally.  We  therefore  equip  the  interface 
of  a  component  with  entry  conditions  and  insist  that  a  location  can  be  hidden 
only  if  the  corresponding  entry  condition  is  valid.  Second,  if  two  components  A 
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and  B  are  composed  in  series,  the  assume-guarantee  principle  is  sound  only  if 
each  trace  of  the  composite  system  A  +  B  can  be  assigned  uniquely  to  either 
A  or  B.  This  can  be  achieved  by  requiring  that  for  all  locations  common  to  A 
and  S,  the  entry  conditions  are  disjoint.  Third,  if  A  and  B  are  composed  in  par¬ 
allel,  we  wish  to  model  the  fact  that  either  component  may  preempt  the  other 
on  termination,  causing  A\\B  to  terminate.  Therefore,  in  refinement,  B  is  more 
specific  than  C  not  only  if  every  trace  of  B  is  a  trace  of  C,  but  also  if  every 
trace  of  B  has  a  prefix  (possibly  generated  if  B  is  preempted)  which  is  a  trace  of 
C.  This  novel  notion  of  refinement  is  consistent  with  sequential  composition:  a 
trace  may  terminate  at  an  exit  location  of  a  component,  and  the  serial  addition 
of  another  component  can  then  provide  it  with  a  continuation.  Thus,  a  prefix  of 
a  trace  is  more  general  than  the  trace  itself,  since  it  potentially  allows  several 
different  continuations.  It  will  follow  that  both  parallel  and  serial  composition 
are  congruences  with  respect  to  refinement. 

We  illustrate  our  formalism  by  modeling  at  different  levels  of  detail  a  sys¬ 
tem  of  two  cooperating  robots,  one  of  which  is  always  following  the  other.  The 
specification  requires  that  a  request  by  one  robot  to  lead  is  honored  within  a 
certain  time  bound  by  the  other  robot  starting  to  follow.  We  give  an  imple¬ 
mentation  that  relies  on  periodic  sampling  of  the  robot  states,  and  show  how 
assume-guarantee  reasoning  simplifies  the  task  of  refinement  checking  between 
implementation  and  specification. 


Related  work.  Concurrent  and  sequential  hierarchies  have  long  been  nested  in 
informal  and  semiformal  ways  (for  instance,  Statecharts  [Har87],  UML  [BRJ98], 
Ptolemy  [DGH''‘99]).  While  these  languages  enjoy  considerable  acceptance  as 
good  engineering  practice,  the  most  widely  used  versions  of  these  languages  do 
not  support  compositional  formal  analysis.  For  Statecharts,  variants  with  compo¬ 
sitional  semantics  have  been  defined  (see,  e.g.,  [US94]),  but  an  assume-guarantee 
paradigm  is  not  known.  Hierarchic  Modules  [AGOO]  provide  an  assume-guarantee 
principle  for  serial  composition,  and  parallel  composition  is  reduced  to  serial  com¬ 
position.  No  continuous  behaviors  are  considered.  The  languages  Shift  [DGV97] 
and  Charon  [AGH+OO]  support  the  hierarchical  design  of  hybrid  systems,  but  its 
emphasis  is  on  simulation,  and  serial  and  parallel  composition  cannot  be  nested 
arbitrarily.  The  model  of  Hybrid  I/O  Automata  [LSVW96]  offers  composition- 
ality  in  a  setting  without  serial  composition. 


2  The  Masaccio  Model  for  Embedded  Components 

In  Masaccio,  a  system  model  is  built  out  of  components.  We  illustrate  Masaccio 
by  modeling  parts  of  a  system  with  two  communicating  robots,  which  will  be 
used  in  Section  4;  the  formal  definition  of  Masaccio  is  given  in  the  appendix. 
The  semantics  of  a  component  is  defined  by  its  interface  (“structure”)  and  its 
set  of  executions  (“behavior”).  The  executions  are  hybrid:  the  state  of  a  compo¬ 
nent  may  evolve  by  any  sequence  of  discrete  transitions  (so-called  jumps)  and 
continuous  evolutions  (flows). 
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Fig.  1.  Robot  specification 


The  interface  of  a  component.  The  interface  of  a  component  determines 
how  the  component  can  be  composed  (i.e.,  can  interact)  with  other  components. 
In  Masaccio,  control  and  data  are  handled  separately.  The  interface  of  a  com¬ 
ponent  A  contains  a  set  Va  of  variables  partitioned  into  input  variables  and 
output  variables,  and  a  set  La  of  interface  locations,  through  which  control  can 
enter  and/or  exit  the  component.  All  variables  are  typed,  with  domains  such  as 
the  booleans  B,  the  natural  numbers  N,  and  the  reals  R.  While  control  resides 
inside  a  component,  the  input  variables  are  updated  by  the  environment  (such 
as  another  component  put  in  parallel),  and  the  output  variables  are  updated 
by  the  component.  The  component  interface  specifies  a  dependency  relation  -<a 
between  I/O  variables  and  output  variables.  If  x  -<a  y-,  then  the  value  of  y  can 
depend  without  delay  on  the  value  of  x.  Specifically,  with  each  jump,  the  new 
value  of  output  y  can  depend  on  the  new  value  of  (say)  input  x,  and  during  a 
flow,  the  derivative  of  output  y  can  depend  on  the  simultaneous  derivative  of 
input  X.  The  dependency  relation  must  be  acyclic,  in  order  to  guarantee  the 
existence  of  suitable  output  values  and  output  curves. 

An  I/O  state  of  the  component  is  a  value  assignment  to  the  variables  in  Va- 
The  component  interface  specifies  for  each  location  a  G  La  a  jump  entry  com 
dition  and  a  flow  entry  condition  The  component  can  be 

entered  by  a  jump  iff  the  jump  entry  condition  is  satisfied  by  the  current  I/O 
state,  and  by  the  new  values  of  the  input  variables;  the  component  can  be  en¬ 
tered  by  a  flow  iff  the  flow  entry  condition  is  satisfied  by  the  current  I/O  state. 
The  length  of  a  flow  may  be  constrained  by  the  component,  but  whenever  the 
flow  entry  condition  is  satisfied,  at  least  a  flow  of  duration  0  is  possible.  Control 
can  exit  the  component  at  every  location.  In  typical  designs,  the  exit  points  are 
the  locations  with  unsat isfiable  entry  conditions. 

As  an  example,  we  portray  a  scenario  in  which  two  similar  robots,  structured 
as  in  Figure  1,  move  around  in  an  environment  with  obstacles.  The  robots  jointly 
choose  the  strategy  of  one  leading  and  the  other  following,  and  their  roles  can 
switch.  The  interface  of  robot  A  consists  of  five  input  and  six  output  variables.  It 
contains  a  unique  location  cr,  with  jump  entry  condition  true  (not  represented). 
Once  entered,  the  robot  will  react  and  execute  forever,  without  control  exiting. 
The  inputs  leads  and  switch b  indicate  whether  robot  B  is  in  the  lead  mode,  or 
about  to  switch  from  follow  to  lead.  The  input  obstA  indicates  if  an  obstacle  is 
encountered.  The  component  Motors,  shown  in  Figure  2,  controls  the  motion  of 
the  two  wheels  based  on  the  signals  leftj^  and  right  a,  which  allow  the  robot  to 
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Fig.  2.  Motor  specification 


go  straight,  halt,  or  turn  in  either  direction.  The  outputs  xa  and  da  give  the 
position  of  the  robot. 


The  executions  of  a  component.  The  behavior  of  a  component  A  is  described 
by  a  set  Ea  of  fi.nite  executions;  the  treatment  of  infinite  behaviors  for  the  study 
of  liveness  issues,  such  as  nonzenoness  [Hen96],  is  deferred  for  now.  An  execution 
is  either  a  triple  (a,it;,6)  or  a  pair  {a^w)  defined  by  an  origin  location  a  €  La, 
a  nonempty  finite  sequence  w  of  execution  steps  and,  possibly,  a  destination 
location  b  G  An  execution  step  is  either  a  jump  or  a  flow.  A  jump  consists 
of  a  source  I/O  state  and  a  sink  I/O  state;  a  flow  consists  of  a  real  duration 
(5  >  0  together  with  a  differentiable  curve  /  that  maps  every  real  time  in  the 
compact  interval  [0,5]  to  an  I/O  state.  For  types  other  than  R,  we  assume  that 
only  constant  functions  are  differentiable.  The  source  of  the  flow  is  the  I/O 
state  /(O),  and  the  sink  is  f{S).  For  any  two  successive  execution  steps,  the  sink 
of  the  first  must  coincide  with  the  source  of  the  second.  In  figures,  arrows  with 
double  tips  denote  flows,  whereas  normal  arrows  represent  jumps. 

The  set  Ea  of  executions  is  prefix-closed.  Indeed,  if  a  component  permits  a 
flow  of  a  certain  duration,  then  all  restrictions  of  the  flow  to  shorter  durations, 
including  the  restriction  to  duration  0,  are  also  permitted.  Every  component  is 
deadlock-free,  in  the  sense  that  (1)  if  the  jump  entry  condition  of  a  location  a 
is  satisfiable  at  an  I/O  state  q,  then  there  is  an  execution  with  origin  a  which 
starts  with  a  jump  with  source  q,  (2)  if  the  flow  entry  condition  of  location  a  is 
true  at  q,  then  there  is  an  execution  with  origin  a  which  starts  with  a  flow  with 
source  q,  and  (3)  every  execution  that  does  not  end  in  a  destination  location  can 
be  prolonged  by  either  a  destination  or  a  jump.  Indeed,  the  stronger  condition  of 
input-permissiveness  holds,  which  asserts  that  a  component  cannot  deadlock  no 
matter  how  the  environment  decides  to  change  the  inputs,  by  either  jumping  or 
flowing.  Prefix- closure,  deadlock-freedom,  and  input-permissiveness  are  formally 
defined  and  proved  in  the  full  version  of  this  paper.  They  are  essential  properties 
of  every  component,  because  the  environment  (another  component)  may  decide 
to  interrupt  a  flow  at  any  time  to  perform  a  jump,  in  which  case  the  component 
must  be  prepared  to  match  the  environment  jump  by  a  local  jump. 


Atomic  components.  Every  component  in  Masaccio  is  built  from  two  kinds 
of  atomic  components,  with  discrete  and  continuous  behavior,  respectively.  An 
atomic  component  has  an  arbitrary  number  of  input  and  output  variables,  but 
only  two  locations,  which  serve  as  origin  and  destination,  respectively,  for  its 
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executions,  all  of  which  contain  a  single  step.  For  an  atomic  discrete  compo¬ 
nent,  that  step  is  a  jump;  for  an  atomic  continuous  component,  a  flow.  The  legal 
jumps  of  an  atomic  discrete  component  are  defined  by  a  jump  predicate,  which 
constrains  the  output  values  of  the  sink  depending  on  the  source  I/O  state  and 
on  input  values  of  the  sink.  Such  a  predicate  is  typically  specified  by  a  differ¬ 
ence  equation.  The  legal  flows  of  an  atomic  continuous  component  are  defined 
by  a  flow  predicate,  which  constrains  the  time  derivatives  of  output  variables 
depending  on  the  current  I/O  state  and  on  the  current  time  derivatives  of  input 
variables.  Such  a  predicate  is  typically  specified  by  a  differential  equation,  as  in 
Figure  2,  A  flow  predicate  may  also  constrain  the  values  of  output  variables,  so 
that  a  flow  must  not  go  on  for  any  duration  that  would  violate  this  “invariant” 
condition.  Both  jump  predicates  and  flow  predicates  may  allow  nondeterminism. 


Operations  on  components.  Discrete  components  are  built  from  atomic  dis¬ 
crete  components  using  the  six  operations  of  parallel  and  serial  composition,  vari¬ 
able  and  location  renaming,  and  variable  and  location  hiding,  arbitrarily  nested. 
The  discrete  components  conservatively  extend  Reactive  Modules  [AH99]  by  se¬ 
rial  composition.  Hybrid  components  are  built  from  both  discrete  and  continuous 
atomic  components  using  the  same  six  operations. 


Parallel  composition  is  defined  synchronously,  as  conjunction,  with  static  await 
dependencies  between  outputs  and  inputs  preventing  circularity.  For  two  compo¬ 
nents  A  and  B,  an  execution  of  the  parallel  composition  A\\B  starts  at  a  common 
location  in  O  L^.  The  execution  is  synchronous  in  both  components:  each 
jump  of  A  must  be  matched  by  a  concurrent  jump  of  B,  and  each  flow  of  A 
must  be  matched  by  a  concurrent  flow  of  B  with  the  same  duration.  Control 
exits  the  parallel  composition  when  it  exits  any  one  of  the  two  components.  If 
the  execution  of  A  reaches  a  destination  location,  then  the  concurrent  execution 
of  B  is  preempted  and  terminated;  if  B  reaches  a  destination  location,  then  the 
concurrent  execution  of  A  is  terminated;  if  both  A  and  B  simultaneously  reach 
destination  locations,  then  the  result  is  nondeterministic.  When  constructing  a 
parallel  composition  A\\B,  inputs  of  A  can  be  identified  with  outputs  of  B,  and 
vice  versa,  by  renaming  variables.  Such  identifications  are  depicted  by  solid  lines 
in  the  figures.  Similarly,  locations  of  A  can  be  identified  with  locations  of  B  by 
renaming  locations;  these  identifications  are  depicted  by  dotted  lines.  We  write 
A[x  :=  y]  for  the  component  that  results  from  renaming  the  variable  a:  in  A  to  ?/, 
and  A[a  :=  b]  for  the  component  that  results  from  renaming  the  location  a  in  A 
to  b. 

In  Figure  1,  the  component  Robots  is  the  parallel  composition  of  the  com¬ 
ponents  ControU  and  Motors.  Before  composition,  the  two  entry  locations  ec 
and  cmt  are  renamed  to  a  common  location  eR. 


Serial  composition  and  location  hiding  can  be  used  to  achieve  the  sequencing  of 
components.  Serial  composition  represents  disjunctive  choice  between  the  exe¬ 
cutions  of  two  components.  For  two  components  A  and  B,  an  execution  of  the 
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Fig.  3.  Serial  composition  and  location  hiding 

serial  composition  A  +  Bis  either  an  execution  of  A  or  an  execution  of  B.  Hiding 
renders  a  location  internal  to  a  component,  and  inaccessible  (invisible)  from  the 
outside.  The  executions  of  the  resulting  component  are  obtained  by  stringing 
together  at  that  location  any  finite  number  of  executions  of  the  original  com¬ 
ponent.  To  avoid  internal  deadlock,  a  location  a  can  be  hidden  only  if  its  jump 
entry  condition  is  valid,  so  that  it  can  always  take  another  jump  at  a.  We  write 
yl\a  for  the  component  that  results  from  hiding  a  in  A. 

Figure  3  shows  how  a  sequential  component  (representing  the  straight  move¬ 
ment  of  the  robot  in  the  lead  mode)  is  obtained  by  the  serial  composition  of  sev¬ 
eral  components,  followed  by  location  hiding.  Let  Straight^  =  (SI  +  S2  +  S3)\a, 
where  Si  and  S3  are  atomic  discrete  components,  and  S2  is  obtained  from  an 
atomic  continuous  component  by  renaming  destination  location  to  origin  loca¬ 
tion.  The  resulting  component  initializes  its  output  variables  by  a  jump,  flows 
(without  output  changes)  for  any  amount  of  time  as  long  as  the  input  obstA 
remains  false,  and  nondeterministically  exits  with  a  jump.  In  the  same  way, 
any  “automaton  structure”  can  be  built  from  individual  “edges”  (i.e.,  atomic 
components)  using  serial  composition,  location  renaming,  and  location  hiding. 


Variable  hiding  builds  an  abstract  component  by  turning  some  outputs  of  a 
component  into  internal  state.  Hidden  variables,  however,  do  not  maintain  their 
values  from  one  exit  of  a  component  to  a  subsequent  entry,  but  they  are  nonde¬ 
terministically  reinitialized  upon  every  entry  to  the  component  so  as  to  satisfy 
the  applicable  entry  condition.  We  write  A\x  for  the  component  that  results 
from  hiding  the  output  variable  x  of  the  component  A. 

3  Assume-Guarantee  Refinement  between  Components 

If  component  A  refines  component  H,  then  B  can  be  viewed  as  a  more  abstract 
(permissive)  version  of  A,  with  some  details  (constraints)  left  out  in  B  which 
are  spelled  out  in  A.  In  particular,  in  the  trace-based  semantics  of  concurrent 
systems,  refinement  is  taken  to  be  the  containment  relation  on  trace  sets.  If  A 
refines  J5,  then  A  is  a  more  specific  description  of  system  behavior  than  B  in 
the  sense  that  A  may  be  equivalent  to  H||C  for  some  parallel  context  C  which 
constrains  the  inputs  to  B.  In  analogy,  in  the  trace-based  semantics  of  sequential 
systems,  refinement  ought  to  be  interpreted  as  prefix  relation  on  trace  sets.  If 
A  refines  H,  then  A  is  a  more  specific  description  of  system  behavior  than  B  in 
the  sense  that  A  may  be  equivalent  to  J5  +  C  for  some  serial  context  C  which 
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constrains  the  continuations  of  B.  Consequently,  in  Masaccio,  if  A  refines  B, 
then  A  may  specify  fewer  traces  and  longer  traces  than  B. 


The  refinement  relation.  Component  A  refines  component  B  if  the  following 
two  conditions  are  satisfied: 

1 .  Every  output  variable  of  B  is  an  output  variable  of  A,  every  input  variable 
of  B  is  an  I/O  variable  of  A,  and  the  dependency  relation  of  5  is  a  subset 
of  the  dependency  relation  of  A. 

2.  For  every  execution  {a,w)  (or  {a,w,b),  respectively)  of  A,  either  {a,w[VB]) 
(or  {a,w[VB],b),  respectively,  where  w\yB\  is  the  projection  of  w  to  the 
variables  of  B)  is  an  execution  of  J5,  or  there  exist  a  proper,  nonempty 
prefix  w'  of  w  and  an  interface  location  c  £  Lb  such  that  {a,w'[VB],c)  is 
an  execution  of  B, 

Note  that  the  second  condition  implies  that  every  interface  location  of  A  is  an 
interface  location  of  B,  Furthermore,  by  input-permissiveness,  if  A  refines  jB, 
then  for  every  location  a  of  A,  the  jump  entry  condition  of  a  in  A  implies  the 
jump  entry  condition  of  a  in  B,  and  the  flow  entry  condition  of  a  in  A  implies 
the  flow  entry  condition  of  a  in  5. 


Compositionality.  All  six  operations  on  components  are  compositional. 

Theorem  1.  Let  A  and  B  be  components,  let  x  and  y  be  variables,  and  let  a 
and  b  be  locations  so  that  the  following  expressions  are  all  well-defined.  If  A 
refines  B,  then  A\\C  refines  B\\C;  and  A  +  C  refines  B  +  C;  and  A[x  :=  y] 
refines  B[x  :=  y];  and  A[a  :=  6]  refines  B[a  :=  b];  and  A\x  refines  B\x;  and 
A\a  refines  B\a. 

More  generally,  define  a  context  to  be  a  component  expression  that  can  take  a 
component  as  a  parameter.  For  instance,  if  (A  B)\\D  is  well-defined,  we  can 
regard  C[']  =  ([•]  -j-  B)\\D  as  a  context  for  component  A. 

Corollary  1.  Let  C[-]  be  a  context  for  both  Ai  and  A2.  If  Ai  refines  A2,  then 
C[Ai],  refines  C[A2]. 

Assume-guarantee  reasoning.  Our  assume-guarantee  rule  states  that  for  dis¬ 
crete  components,  if  two  components  can  be  individually  replaced  in  a  context 
while  maintaining  refinement,  then  both  can  be  replaced  simultaneously.  There¬ 
fore,  in  order  to  show  that  a  complex  component  C[Ai,Bi]  (the  “implementa¬ 
tion”)  refines  a  simpler  component  C[A2,B2]  (the  “specification”),  it  suffices  to 
look  at  simplified  versions  of  the  implementation  one  at  a  time.  First,  we  prove 
that  Ai  refines  its  specification  Bi,  under  the  “assumption”  B2]  then,  we  prove 
that  A2  refines  its  specification  B2,  under  the  “assumption”  Bi.  This  reason¬ 
ing  is  inherently  circular.  A  special  case  is  the  assume-guarantee  rule  for  the 
parallel  composition  of  Reactive  Modules  [AH99]:  take  the  context  C[o,  •]  in  the 
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following  theorem  to  be  o|  j*.  The  proof  relies  on  the  deadlock-freedom  and  input¬ 
permissiveness  of  components.  It  also  requires  that  each  execution  of  a  serial  com¬ 
position  can  be  uniquely  assigned  to  one  of  the  components.  This  can  be  achieved 
by  disjoint  entry  conditions.  We  say  that  the  serial  composition  A B  is  jump- 
deterministic  if  for  all  common  interface  locations  a  G  H  the  conjunction 
^^ump(a)  is  unsatisfiable,  and  flow- deterministic  (a)  (a) 

is  unsatisfiable  for  all  a  E  La^Lb-  The  serial  composition  A +5  is  deterministic 
if  it  is  both  jump-deterministic  and  fiow-deterministic. 

For  hybrid  modules,  we  need  to  break  the  circularity  of  the  rule,  by  relaxing 
one  assumption,  say,  B2,  to  allow  arbitrary  fiows  at  all  hidden  locations.  We 
write  rlax{B2)  for  the  component  that  results  from  B^  by  (1)  replacing  every 
flow  predicate  in  B2  by  true,  and  (2)  serially  composing  every  hidden  location 
a  of  B2  which  is  not  the  origin  location  of  any  fiow,  with  an  atomic  continuous 
component  that  permits  all  flows  from  origin  a  to  destination  a. 

Theorem  2.  Let  Clo,*]  be  a  context  whose  arguments  are  not  in  the  scope  of 
any  variable  or  location  hiding.  Suppose  that  all  input  variables  of  C\A2-,B2\  are 
variables  ofC[Ai,Bi],  and  that  within  C[A2,B2\  the  context  arguments  are  not 
within  the  scope  of  any  nondeterministic  serial  composition.  If  C[Ai,rlax{B2)] 
refines  C[A2,rlax{B2)],  andC\A2,Bi]  refines  C[A2,B2],  then  C[Ai,Bi]  refines 
C[A2,B2]. 

Linear  components.  If  all  flows  are  specified  by  linear  differential  equations, 
and  no  degenerate  flows  of  0  duration  can  be  enforced,  then  the  existence  of 
unique  solutions  allows  us  to  strengthen  the  assume-guarantee  rule.  In  this  case, 
we  can  make  circular  assumptions  about  the  flows.  An  open  linear  condition  on  a 
set  V  of  real- valued  variables  is  a  conjunction  of  boolean  variables  and  strict  (< 
or  >)  comparisons  between  linear  combinations  of  the  variables  in  V.  Consider 
a  flow  action  F  (consult  the  appendix  for  a  definition).  The  atomic  continuous 
component  A(F)  is  linear  if  (1)  all  variables  in  Va{F)  have  the  type  M,  and 
(2)  the  flow  predicate  9?^^^  has  the  form  a{XF)  A  (Zp  =  fliXp,  Fjp)),  where  a  is 
an  open  linear  condition,  called  invariant,  on  the  source  variables  Xp,  and  /?  is  a 
set  of  linear  combinations,  one  for  the  derivative  z  e  Zp  of  each  controlled  flow 
variable,  of  the  source  variables  Xp  and  the  derivatives  Yp  of  the  uncontrolled 
flow  variables.  A  component  is  linear  if  (1)  all  its  atomic  continuous  components 
are  linear,  and  (2)  all  its  serial  compositions  are  flow-deterministic.  Let  rlax'  be 
defined  like  rlax,  with  the  difference  that  only  the  invariants  rather  than  the 
flow  predicates  are  replaced  with  true. 

Theorem  3.  Let  C[o,  •]  be  a  context  whose  arguments  are  not  in  the  scope  of 
any  variable  or  location  hiding.  Suppose  that  C[Ai,Bi]  and  C[A2yB2]  are  linear 
components,  that  all  input  variables  of  C[A2,B2]  are  variables  of  C[Ai,Bi], 
and  that  within  C[A2,B2\  the  context  arguments  are  not  within  the  scope  of  any 
nondeterministic  serial  composition.  If  C[Ai,  rlax' {B2)]  refines  (7[A2,  rlax  {B2)], 
and  C[A2,Bi]  refines  C[A2,B2],  then  C[Ai,Bi\  refines  C[A2,B2]- 
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Fig.  5.  Components  Straight^  and  Turriyi 


4  A  Two- Robot  Example 

We  continue  the  presentation  of  the  two-robot  system  whose  overall  view  was 
given  in  Section  2.  Robot  A  (Figure  1)  starts  out  as  the  leader.  After  a  while 
it  may  move  from  Lead^  to  FoIlow>i,  as  indicated  by  the  dotted  line  connecting 
location  X£  (with  an  unsatisfiable  entry  condition,  which  is  not  shown)  and  lo¬ 
cation  ejr.  It  may  then  move  back  to  lead  mode  (line  XF-eL2)-  Robot  B  has  the 
same  structure,  except  that  it  starts  out  in  follow  mode.  W^ithin  the  subcom¬ 
ponent  Move^  (Figure  4),  the  robot  can  execute  in  Straight^  arbitrarily  long 
while  there  is  no  obstacle.  Upon  sensing  an  obstacle,  control  is  passed  to  the 
component  Turn^,  which  commands  the  robot  to  rotate  for  an  amount  of  time 
given  by  timer  variable  cIHa-  Control  then  returns  to  the  component  Straight^, 
The  sequence  of  straight  moves  and  turns  continues  until  robot  B  switches  to 
leading  status.  This  event  is  modeled  by  the  boolean  signal  switch q,  which  is 
monitored  by  the  component  Switcher^i.  We  require  the  switcher  unit  to  preempt 
execution  of  the  lead  mode  within  a  specified  amount  of  time  Tsw  after  the  other 
robot  has  signaled  its  intention  to  lead.  Once  Lead^i  is  exited,  control  enters  the 
component  Follow^,  which  samples  the  values  of  lefts  and  rights  and  drives 
its  own  motor  signals  left^  and  right The  robot  may  stay  in  the  follow  mode 
arbitrarily  long,  provided  that  ohstA  is  false.  At  any  time  it  may  also  issue  the 
signal  switch  A,  exit  the  component  FolloWy^  and  switch  back  to  lead  mode. 

We  now  present  a  robot  implementation  that  contains  a  modified  component 
Lead^,  which  does  not  continuously  observe  the  switch  signal  (Figure  7).  Instead, 
the  implementation  samples  the  leading  indicators  of  both  robots  with  a  period 
Ted,  measured  by  the  global  clock  elk.  If  both  robots  are  leading,  a  correction 
is  made  by  the  component  Errordetect^.  The  new  state  depends  on  the  last 
sampled  values  of  the  leading  signals:  the  robot  that  had  been  leading  before 
now  switches  to  follow  mode. 

We  wish  to  show  that  when  composed  together,  two  robot  implementations 
refine  the  parallel  composition  of  two  robot  specifications,  provided  that  T^d  < 
Tsw  The  specification  of  robot  A  is  Control^ || Motors,  and  the  implementation 
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Fig.  6.  Components  Switcher^  and  Follow^ 

of  robot  A  is  Control^  || Motors,  where  Control  =  (Lead^  +  Follow^)\ei,2\eF  and 
Control^  =  (Lead^  +  FolloWA)\eL2\eF-  Robot  B  is  specified  and  implemented 
symmetrically.  Denoting  the  parallel  composition  with  the  motor  by  the  context 
C^[.]  =  .|  I  Motors,  and  similarly  for  Cb,  we  wish  to  prove  that 

CAlControl^JlICBlControlg]  refines  CAlControUJIlCsIControlF]. 

Note  that  Ca [Control^]  does  not  refine  Ca [Control a],  because  a  robot  imple¬ 
mentation  meets  the  specification  only  when  composed  with  a  symmetric  robot. 
This  is  where  assume-guarantee  reasoning  helps.  All  continuous  components  in 
the  system  are  linear.  Hence  by  Theorem  3,  it  suffices  to  discharge  the  simpler 
assertions 

CAlControlAlllC'BlControl^]  refines  C7A[ControlA]||C'B[ControrB] 

Ca  [Control  a1  I  [CBiControlB]  refines  CA[ControlA]iiCB[ControlBi, 
where  Control'^  =  rlax' {Control b)^  We  simplify  further  using  compositionality 
(Theorem  1),  and  are  left  to  prove  that 

ControlA  II  Control'^  refines  ControU  ||  Control'^ 

Control  A  II  Control^  refines  Control  ||  Controls, 
two  proof  obligations  that  involve  simpler  components  than  the  original  one. 
The  power  of  the  assume-guarantee  rules  of  Theorems  2  and  3  stems  fi:om  the 
fact  that  they  can  be  applied  to  components  arbitrarily  deep  in  the  design  hier¬ 
archy,  creating  proof  obligations  which  have  smaller  differences  between  the  two 
components  which  are  supposed  to  refine  each  other. 
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Appendix:  Formal  Definition  of  Masaccio 

Let  V  be  a  set  of  typed  variables.  For  a  variable  x£V,  denote  by  x'  its  primed 
version,  and  denote  by  x  its  dotted  version.  The  type  of  x^  is  the  same  as  the 
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type  of  X.  The  type  of  i  is  R  if  the  type  of  x  is  R,  and  {0}  otherwise.  This 
is  because  on  types  other  than  R,  we  assume  that  only  the  constant  functions 
are  differentiable.  Let  V'  =  {x'  \  xeV}  be  the  set  of  primed  versions  of  the 
variables  in  V,  and  let  V  ^  {x\x  eV}  be  the  set  of  dotted  versions  of  the 
variables  in  V,  Let  [V]  be  the  set  of  type-conforming  value  assignments  to  the 
variables  in  V:\txeV  and  q  G  [V],  let  q{x)  be  the  value  assigned  by  q  to  x. 

The  interface  of  a  component.  The  interface  of  a  component  A  consists  of: 

—  A  finite  set  V\  of  typed  input  variables. 

—  A  finite  set  of  typed  output  variables,  such  that  n  =  0.  Let  Va  ~ 

u  VX  be  the  set  of  I/O  variables.  The  value  assignments  in  [V^]  are  called 
I/O  states. 

—  An  dependency  relation  -<a  Q  Va  x  between  I/O  variables  and  output 
variables,  such  that  the  transitive  closure  is  asymmetric.  A  set  t/  C  Va 
of  I/O  variables  is  dependency- closed  if  x  -<a  y  and  y  €U  implies  x  eU. 

—  A  finite  set  La  of  interface  locations.  ^ 

—  For  each  location  a  £  La,  a  predicate  (a)  on  the  variables  in  VaOVX, 

called  jump  entry  condition,  and  a  predicate  variables  in  Va, 

called  flow  entry  condition. 


The  executions  of  a  component.  A  jump  of  a  component  A  is  a  pair  (p,  q)  £ 
[Va]"^  of  I/O  states.  The  I/O  state  p  is  the  source  of  the  jump,  and  q  is  the 
sink.  A  flow  of  A  is  a  pair  (6,  f)  consisting  of  a  nonnegative  real  5  G  R>o,  and  a 
function  /:  R  [Va]  from  the  reals  to  I/O  states  which  is  differentiable,  with 
time  derivative  on  the  compact  interval  [0,  d]  C  R.  The  real  5  is  the  duration 
of  the  flow,  the  I/O  state  /(O)  is  the  source,  and  f{S)  is  the  sink.  A  step  of  A  is 
either  a  jump  or  a  flow  of  A.  The  step  w  is  successive  to  the  step  v  if  the  sink  of 
v  is  equal  to  the  source  of  w.  An  execution  of  A  is  either  a  pair  (a,  w)  or  a  triple 
(a, w,h),  where  a,b  e  La  are  interface  locations,  and  w  =  wo'-Wn  is  a  finite, 
nonempty  sequence  of  steps  of  A  such  that  (1)  every  step  Wi,  for  1  <  i  <  n,  is 
successive  to  the  preceding  step  Wi^i,  and  (2)  the  first  step  wq  satisfies  the  entry 
conditions  of  location  a:  if  wq  =  {p,q)  is  a  jump,  then  is  true  if  each 

I/O  variable  a:  G  is  assigned  the  value  p{x),  and  each  primed  input  variable 
y'  G  Vj^  is  assigned  the  value  q{y);  if  wq  =  {S,f)  is  a  flow,  then  is  true 

if  each  I/O  variable  x  €  Va  is  assigned  the  value  f{0){x).  The  location  a  is  the 
origin  of  the  execution,  the  sequence  w  is  the  trace,  and  the  location  b  (when 
present)  is  the  destination.  Given  a  trace  w  and  a  set  1/  C  Va  of  I/O  variables, 
we  write  w[U]  for  the  projection  of  w  to  the  variables  in  U, 

Atomic  discrete  components.  An  atomic  discrete  component  is  specified  by 
a  jump  action.  A  jump  action  J  consists  of  a  finite  set  Xj  of  source  variables, 
a  finite  set  Yj  of  uncontrolled  sink  variables,  a  finite  set  Zj  of  controlled  sink 
variables  disjoint  from  Yj,  and  a  predicate  on  the  variables  in  XjUyjU.^j, 

called  jump  predicate.  The  jump  action  J  specifies  the  component  A(J).  The 
interface  of  the  component  A(  J)  is  defined  as  follows: 
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-  Kw  =  Zj. 

-  y  -<A{J)  Z  iff  2/  6  Fj  and  2:  €  Zj. 

-  La(J)  =  {from,  to). 

-  i^A^^ifrom)  =  (^Z'j)  and  =  false. 

(to)  =  false. 

The  executions  of  the  component  A{J)  are  defined  as  follows.  The  pair  (a,  w)  is 
an  execution  of  A{J)  iff  a  =  from  and  the  trace  w  consists  of  a  single  jump  (p,  q) 
such  that  is  true  if  each  source  variable  x  G  Xj  is  assigned  the  value  p{x) , 

and  each  primed  sink  variable  y'  eY}U  Z'j  is  assigned  the  value  q{y).  The  triple 
(a,  w,  b)  is  an  execution  of  A{J)  iff  the  pair  (a,  w)  is  an  execution  of  A{J),  and 


Atomic  continuous  components.  An  atomic  continuous  component  is  spec¬ 
ified  by  a  flow  action.  A  flow  action  F  consists  of  a  finite  set  Xp  of  source 
variables^  a  finite  set  Yp  of  uncontrolled  flow  variables,  a  finite  set  Zp  of  con¬ 
trolled  flow  variables  disjoint  from  Yp,  and  a  predicate  on  the  variables 

in  XpUYpU  Zp,  called  flow  predicate.  The  flow  action  F  specifies  the  compo¬ 
nent  A{F).  The  interface  of  the  component  A{F)  is  defined  as  follows: 

-  VX^^^  =  {Xp\Zp)uYp. 

~  ^A(F)  — 

~  y  -<A{F)  z\Q.y^Yp  and  z  e  Zp. 

~  ^A{F)  =  {from,  to}. 

-  '^Mnif^om)  =  false  and  iP^^^^^{from)  =  {3Yp,  Zp) 

The  executions  of  the  component  A{F)  are  defined  as  follows.  The  pair  {a,  w) 
is  an  execution  of  A(F)  iff  a  =  from  and  the  trace  w  consists  of  a  single  flow 
{6,f)  such  that  the  following  holds:  ii  S  =  0,  then  {3Yp,  Zp)  is  true  if 
each  source  variable  x  e  Xp  is  assigned  the  value  /(0)(a:);  if  5  >  0,  then  for 
all  £  e  [0,5],  the  flow  predicate  is  true  if  each  source  variable  x  ^  Xp  is 
assigned  the  value  f{£)(x),  and  each  dotted  flow  variable  y  G  YpUZp  is  assigned 
the  value  f{£){y).  The  triple  (a,  w,  b)  is  an  execution  of  A{F)  iff  the  pair  (a,  w) 
is  an  execution  of  A{F),  and  b=  to. 


Parallel  composition.  Two  components  A  and  B  can  be  composed  in  parallel 
if  their  interfaces  satisfy  the  following  conditions: 

-  n  Vg  =  0. 

-  There  are  no  two  variables  r  G  and  x/  e  such  that  both  r  ?/  and 
y-<A 
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-  For  all  a  €  if  or  is  satisfiable,  then  a  e  Lb-  For 

all  a  G  Lb,  if  or  is  satisfiable,  then  a  e  L^-  For  all 

a  e  La  ^  Lb,  the  projections  of  the  entry  conditions  of  a  in  A  and  B  to 
the  common  variables  are  equivalent:  (3Va  \  Vb)(3V^  \  is 

equivalent  to  (3Vb  \  K4)(3V^'  \  ^4  ) and  (3^4  \  Vg)  is 

equivalent  to  {3Vb  \  Va)  ipg‘‘'“(a). 

The  interface  of  A\\B  is  defined  from  the  interfaces  of  A  and  B\ 

-  vX\\B  =  in\v§)u{v^\vx). 

-  V'lliB  =  vx  u  Vi. 

- <A\]B  =  u  -<B. 

~  La\\b  =  La^  Lb- 

-  If  a  €  La  n  Lb,  then  i/’a’[|b’(“)  =  i’A^'^^ia)  A  (a)  and  •4>a^b(°‘)  = 

A  If  n  e  La  \  Lb  or  a  e  Lb  \  La,  then  = 

V’Anfi(“)  = 

The  executions  of  A\\B  are  defined  from  the  executions  of  A  and  B.  The 
pair  [a^w)  is  an  execution  of  A\\B  iff  {a^w\VA])  is  an  execution  of  A  and 
(a,ty[yB])  is  an  execution  of  B.  The  triple  {a,w^b)  is  an  execution  of  A||L 
iff  either  {a,w[VA],b)  is  an  execution  of  A  and  {a,w[VB])  is  an  execution  of  J5, 
or  (a,w[VB],b)  is  an  execution  of  B  and  (a,ti)[V^])  is  an  execution  of  A, 

Serial  composition.  Two  components  A  and  B  can  be  composed  in  series  if 
V^  =  Vq.  The  interface  of  A  +  B  is  defined  from  the  interfaces  of  A  and  B: 

-vl+B  =  vxuvi. 

-  vxab  =  yi  =  Vi- 

- <A+B  =  G  -<B- 

-  La+b  =  La  U  Lb- 

-  If  a  €  La  n  Lb,  then  V  V'g‘’”'’(a)  and  V'^';^(a)  = 

V  If  a  G  La  \  Lb,  then  and 

t/'^+B(a)  =  lia  e  Lb  \  La,  then  and 

The  executions  of  A  +  B  are  defined  from  the  executions  of  A  and  B.  The  pair 
{a,w)  is  an  execution  of  A  +  5  iff  either  (a,ty[VA])  is  an  execution  of  A,  or 
(a,  w[Vb])  is  an  execution  of  B.  The  triple  (a,  w,  b)  is  an  execution  of  A  +  B  iff 
either  (a,  ii;[V^],6)  is  an  execution  of  A,  or  {a,w[VB],b)  is  an  execution  of  B. 


Variable  renaming.  The  variable  a:  G  Va  can  be  renamed  to  y  in  component 
A  \i  y  has  the  same  type  as  x,  and  either  y  is  not  an  I/O  variable  of  A,  or  x 
and  y  are  both  input  variables;  that  is,  if  ?/  €  I/4,  then  x,y  ^  V\.  The  interface 
of  the  component  A[x  :=  y]  is  defined  from  the  interface  of  A.  If  x  G  V^,  then 
yi[x,=vl  =  (y4\{^^})uM  and  =  y?;  if  a:  6  y|,  then  =  V*  and 
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^A[x:^y]  =  i^A  \  {^})  ^  {v) ^  either  case,  let  LA[x-.=y]  =  La,  and  let  ^A[x:=y] 
and  fp^Ai^^y]  result  from  renaming  x  to  y,  and  x'  to  y',  in  and 

and  respectively.  The  executions  of  the  component  A[x  :=  y]  result 

from  renaming  j:  to  y  in  the  traces  of  the  executions  of  A. 


Location  renaming.  The  interface  location  a  G  La  can  be  renamed  to  b  in 
component  A  if  either  b  is  not  an  interface  location  of  A,  or  the  entry  conditions 
of  a  and  b  are  disjoint;  that  is,  if  6  e  La,  then  both  A  and 

{a)  [b)  are  unsatisfiable.  The  interface  of  the  component  A[a  :=  b] 

is  defined  from  the  interface  of  A:  let  =  V\;  let  ' 

^>i[a:=6]  =  -<a;  let  LAia:^t]  =  {La  \  {«})  U  {6};  let 

fjumpdjs^ - j  _iJlow  \/  ^tjlow (lA  xf  U  ^  r  1..^  ^iJump 


and  (6)  =  V^; 


if) 


let  =  Va-,  let 

(a)  V 

(6)  if  b  e  La,  let  = 


(a)  if  fc  ^  and  let  V'^'|™£w(c; 


'^A 


jump 


ic) 


'---(a)  and  ,|(f>)  =  « 

and  'pA[<^=b]i^)  ~  all  locations  c  e  La  \  {a,  6}.  The  executions  of  the 

component  A[a  :=  b]  result  from  renaming  a  to  6  in  the  origins  and  destinations 
of  the  executions  of  A. 


Variable  hiding.  The  variable  x  G  Va  can  be  hidden  in  the  component  A  if 
a:  G  V^.  The  interface  of  the  component  A\x  is  defined  from  the  interface  of  A: 

^A\x  —  1®^  ^A\x  =  \  {^};  -<A\x  be  the  intersection  of  the  transitive 

closure  with  x  let  La\.  =  La',  let  {3x)  (a) 

and  i>A\xi^)  ~  'Pa''^ {o)  for  all  locations  a  €  La-  The  executions  of  the 

component  A\x  are  defined  from  the  executions  of  A.  The  pair  {a,w)  is  an 
execution  of  A\x  iff  (a,  is  an  execution  of  A.  The  triple  {a,w,b)  is  an 

execution  of  A\a:  iff  {a,w[VA\x],b)  is  an  execution  of  A. 


Location  hiding.  The  interface  location  c  G  La  can  be  hidden  in  the  compo¬ 
nent  A  if  the  jump  entry  condition  'ip^^'^^(c)  is  equivalent  to  true.  The  interface 
of  the  component  A\c  is  defined  from  the  interface  of  A:  let  =  Vj;  let 

^X\c  =  ^A'^  let  let  La\c  -  La  \  {c};  let  and 

Pa\c  (^)  ~  pA°^ip)  all  locations  a  G  La\c‘  The  executions  of  the  component 
A.\c  are  defined  from  the  executions  of  A.  The  pair  (a,  w)  is  an  execution  of 
A\c  iff  c  ^  a  and  either  (a,  w)  is  an  execution  of  A,  or  there  is  a  finite  sequence 
'^1,  • ,  "Wn  of  traces,  n  >  2,  such  that  w  —  wi  •  •  •  Wn  and  the  following  are 

executions  of  A:  the  triple  (a, iui,c),  the  triples  {c,Wi,c)  for  1  <  i  <  n,  and 
the  pair  {c,w^).  The  triple  (a,iy,6)  is  an  execution  of  A\c  iff  c  ^  {a,  6}  and 
(a,  w,  b)  is  an  execution  of  A,  or  there  is  a  finite  sequence  wi,. ..  ,Wn  of  traces, 
n  >  2,  such  that  w  =  Wi  ■  •  •  and  the  following  are  executions  of  A:  the  triple 
(a,'«;i,c),  the  triples  {c,Wi,c)  for  1  <  i  <  n,  and  the  triple  {c,Wn,b). 
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Abstract.  In  this  paper  we  propose  a  hybrid  model  for  TCP’s  conges¬ 
tion  control  mechanism  operating  under  drop-tail  queuing  policy.  Using 
this  model  we  confirmed  the  standard  formula  T  :=  — used  by 
TCP-friendly  congestion  control  algorithms,  which  relates  the  average 
packet  drop  rate  p,  the  average  round-trip  time  RTT,  and  the  average 
throughput  T.  The  hybrid  model  also  allows  us  to  understand  the  tran¬ 
sient  behavior  and  theoretically  predict  the  flow  synchronization  phe¬ 
nomena  that  have  been  observed  in  simulations  and  in  real  networks 
but,  to  the  best  of  our  knowledge,  have  not  been  theoretically  justified. 
This  model  can  also  be  used  to  detect  abnormalities  in  TCP  traffic  flows, 
which  has  important  applications  in  network  security. 


1  Introduction 

Consider  the  computer  network  shown  in  Figure  1.  In  this  topology,  n  TCP  flows 
are  generated  at  a  source  node  ni  and  are  directed  towards  a  sink  node  n2.  All 
the  flows  compete  for  the  finite  bandwidth  B  that  characterizes  the  link  i  that 
connects  the  nodes.  This  configuration  is  known  as  a  dumbbell  topology  and  is 
typically  used  to  analyze  TCP’s  congestion  control.  In  more  realistic  networks, 
a  path  of  several  links  (and  intermediate  nodes)  would  connect  the  source  and 
destination  nodes.  However,  to  analyze  congestion  control  mechanisms,  one  of¬ 
ten  ignores  the  existence  of  all  the  intermediate  links,  except  for  the  bottleneck 
link,  i.e.,  the  link  that  has  the  smallest  bandwidth.  In  the  dumbbell  topology,  i 
represents  precisely  this  link. 

The  basic  problem  in  congestion  control  is  to  determine  sending  rates  for  each 
of  the  n  flows  that  result  in  an  optimal  utilization  of  the  available  bandwidth, 
avoiding  a  catastrophic  collapse  under  very  heavy  load.  The  transport  layer  of 
the  TCP/IP  protocol  stack  is  responsible  for  solving  this  problem  and  the  send¬ 
ing  rates  are  determined  by  n  congestion  controllers.  Each  congestion  controller 
adjusts  the  sending  rate  of  one  particular  flow,  based  on  the  number  of  packet 
drops  that  this  flow  is  suffering.  Packet  drops  occur  when  the  sending  rates  of 
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and  the  Office  of  Naval  Research.  The  views  presented  here  are  those  of  the  authors 
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Fig.  1.  Dumbbell  topology 


the  flows  are  too  large  and  the  source  node  ni  is  unable  to  process  all  the  packets 
received.  The  congestion  controller  becomes  aware  of  packet  drops  because,  each 
time  a  packet  is  received  by  the  destination  node,  it  sends  an  acknowledgment 
packet  back  to  the  source  node.  When  a  data  packet  is  dropped,  its  acknowledg¬ 
ment  is  never  received  and  the  congestion  controller  should  take  some  action. 
The  congestion  control  problem  is  nontrivial  because  of  the  following: 

1,  The  bandwidth  B  associated  with  the  link  i  and  the  total  number  of  flows  n 
competing  for  this  bandwidth  are  not  known  by  the  congestion  controllers. 
Moreover,  these  parameters  are  likely  to  change  over  time. 

2.  The  exchange  of  information  among  congestion  controllers  and  between  the 
congestion  controllers  and  the  nodes  is  undesirable.  This  is  because  the  con¬ 
trol  information  would  compete  with  the  data  for  the  available  bandwidth. 

Every  computer  connected  to  the  Internet  runs  some  version  of  TCP  con¬ 
gestion  control.  It  is  therefore  not  surprising  to  And  that  a  significant  body  of 
literature  is  devoted  to  this  topic.  However,  many  basic  questions  remain  poorly 
understood.  These  include: 

1.  Does  TCP  congestion  control  work?  In  particular,  is  it  able  to  prevent  a 
catastrophic  collapse  of  the  network  under  very  heavy  load. 

2.  Is  TCP  congestion  control  fair?  In  particular,  does  it  result  in  approximately 
equal  throughput  for  all  competing  flows. 

3.  Is  TCP  optimal  or  close  to  optimal?  This  question  is  particularly  difficult 
because  there  is  no  universally  accepted  notion  of  optimality.  Small  drops 
rates,  small  delays,  approximately  constant  flow  rates,  and  fast  adaptation 
to  changes  in  the  network  are  certainly  desirable  properties.  However,  these 
criteria  are  self-contradictory  and  therefore  trade-off  solutions  are  required. 

In  this  paper  we  provide  a  hybrid  model  for  Reno  congestion  control  [1,2,3]  that 
sheds  light  in  some  of  the  questions  formulated  above.  Reno  is  one  of  the  more 
popular  versions  of  TCP  congestion  control  and  is  generally  accepted  to  perform 
well.  The  model  proposed  also  applies  to  more  recent  variations  on  Reno  such 
as  New  Reno,  Sack  [4],  and  general  AIMD  [5]. 

The  model  proposed  provides  a  new  derivation  for  the  now  fairly  standard 
formula 


RTT^ 


(1) 
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that  relates  the  average  packet  drop  rate  p,  the  average  round-trip  time  RTT , 
and  the  average  throughput  T  [6, 7, 8, 9].  Formulas  such  as  (1)  have  been  used 
to  design  congestion  control  mechanisms  that  are  TCP-friendly  but  produce 
more  constant  sending  rates,  making  them  more  suitable,  e.g.,  for  streaming 
multimedia  over  the  Internet  [10].  Unlike  previous  derivations,  ours  considers 
the  effect  of  queuing  and  the  coupling  between  the  competing  flows. 

The  hybrid  model  presented  here  also  predicts  that  the  dumbbell  topology 
in  Figure  1,  with  drop-tail  queuing  at  node  ni,  leads  to  flow  synchronization, 
i.e.,  the  sending  rates  of  all  the  flows  exhibit  in-phase  periodic  variations.  This 
produces  undesirably  large  variations  of  the  round-trip  time  and  poor  utilization 
of  the  queue.  This  type  of  behavior  has  been  observed  before  [11]  and  actually 
led  to  the  development  of  Random  Early  Detection/Drop  active  queuing  [12, 
13].  To  the  best  of  our  knowledge,  this  is  the  first  time  that  the  synchronization 
phenomena  are  theoretically  explained. 

2  Hybrid  Model  for  Congestion  Control 

In  this  paper,  we  consider  Reno  congestion  control.  We  describe  next  a  simplified 
version  of  this  algorithm  that  is  sufficient  for  the  purposes  of  this  paper.  Each 
congestion  controller  possesses  an  internal  state  known  as  the  window  size.  We 
denote  by  Wi,  i  6  {l,2,...,n},  the  window  size  of  the  congestion  controller 
associated  with  the  ith  flow.  The  window  size  determines  the  maximum  number 
of  unacknowledged  packets  for  that  flow.  E.g.,  if  Wi  =  3,  then  the  congestion 
controller  can  send  3  packets  immediately,  but  must  wait  for  one  acknowledgment 
to  arrive  before  a  4th  packet  can  be  sent.  The  algorithm  to  update  the  window 
size  Wi  is  as  follows:  While  no  drops  occur,  the  window  size  is  incremented  by 
a  fixed  constant  a  >  1  for  each  Wi  acknowledgments  received  (typically  a  =  1). 
This  is  known  as  additive  increase.  When  it  is  detected  that  a  drop  occurred 
(because  an  acknowledgment  packet  is  missing)  the  window  size  is  multiplied 
by  a  constant  m  E  (0,1)  (typically  m  =  1/2).  This  is  known  as  multiplicative 
decrease.  We  are  ignoring  Reno’s  initial  adjustment  of  the  window  size — known  as 
slow  start— because  it  has  little  impact  on  the  system  after  a  brief  initial  period. 
The  reader  is  referred  to  [1,2,3]  for  a  detailed  description  of  Reno  congestion 
control. 

Although  the  window  size  takes  discrete  values,  it  is  convenient  to  regard 
it  as  a  continuously  varying  variable.  Let  us  call  round-trip  time,  denoted  by 
RTT,  the  time  interval  measured  from  the  moment  a  packet  is  sent  until  an 
acknowledgment  for  that  packet  is  received.  As  we  will  see  below,  the  round-trip 
time  is  a  time- varying  quantity.  Suppose  that  at  some  time  t,  the  congestion 
controller  for  the  ith  flow  sends  one  packet  and  fills  its  window.  This  means  that 
Wi  packets  are  now  unacknowledged  for.  Assuming  that  there  are  no  drops,  after 
one  round-trip  time  the  acknowledgment  for  this  packet  is  received,  as  well  as 
the  acknowledgments  for  the  previous  Wi  —  1  packets.  Since  Wi  acknowledgments 
were  received,  the  window  size  must  have  increased  by  a.  On  average,  each  Wi 
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thus  increases  at  a  rate  of  packets  per  second.  The  following  hybrid  model 
provides  a  good  approximation  of  the  ith  window  size  dynamics:  While  the  ith. 
flow  suffers  no  drops  we  have 


RTT’ 

and,  if  a  drop  is  detected  on  this  flow  at  time  we  have  Wi{t)  =  m  where 

w~ (t)  denotes  the  limit  from  below  of  Wi{s)  as  s  1 1. 

We  proceed  to  determine  the  evolution  of  the  round-trip  time  RTT{t),  Typi¬ 
cally,  the  round  trip  time  has  two  components:  a  fixed  propagation  time  Tp  that 
is  determined  by  the  physical  length  of  the  link  £  and  the  speed  of  light,  and 
a  variable  service  time  Tg  that  accounts  for  the  time  the  nodes  take  to  process 
the  packet.  The  service  time  is  usually  dominated  by  the  queue  time  Tq,  i.e.,  the 
time  a  packet  stays  in  the  output  queue  of  node  ni  before  it  is  sent  to  the  link. 
Denoting  by  q{t)  the  size  of  this  queue  at  time  t,  and  by  B  the  bandwidth  of 
link  £  in  packets  per  second,  the  queuing  time  is  given  by 

r,(i)  = 

because  q(t)  packets  need  to  be  transmitted  (each  taking  1/B  seconds)  before  a 
new  packet  can  also  be  transmitted.  We  assume  here  that  the  bandwidth  B  is 
measured  in  packets  per  second.  The  round-trip  time  is  then  given  by 

RTT{t)=T^  +  S!^.  (3) 

In  this  formula,  we  incorporated  in  Tp  any  fixed  component  of  the  service  time. 

As  mentioned  above,  the  ith  flow  receives  Wi  acknowledgment  packets  in 
one  round-trip  time.  Therefore,  in  average,  it  sends  Wi  packets  per  round-trip 
time.  This  means  that  the  output  queue  at  node  ni  receives  a  total  of 
packets  per  second  and  is  able  to  send  B  packets  to  the  link  in  the  same  period. 
The  difference  between  these  two  quantities  determines  the  evolution  of  q{t).  In 
particular, 


Otherwise 

The  first  branch  in  (4)  takes  into  account  that  the  queue  size  cannot  become 
negative  nor  should  it  exceed  the  maximum  queue  size  g'max-  When  q{t)  reaches 
9max  drops  occur.  These  will  be  detected  by  the  congestion  controllers  some  time 
later. 

To  complete  our  model  it  remains  to  understand  how  many  drops  occur  and 
in  which  flows.  As  mentioned  above,  drops  will  occur  whenever  q  reaches  the 
maximum  queue  size  g^ax  and  the  rate  of  incoming  packets  to  the  queue 
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exceeds  the  rate  B  of  outgoing  packets.  Since  a  drop  will  only  be  detected  after 
one  round-trip  time,  the  rate  of  incoming  packets  will  not  change  for  a  period 
of  length  RTT  and  multiple  drops  are  expected.  It  turns  out  that,  in  most 
operating  conditions,  exactly  one  drop  per  flow  will  occur  [11].  To  understand 
why,  we  must  recall  that  in  every  round-trip  time  the  window  size  of  each  flow  will 
increase  because  each  flow  will  receive  as  many  acknowledgments  as  its  window 
size.  When  the  acknowledgment  that  triggers  the  increase  of  the  window  size  by 
a  >  1  arrives,  the  congestion  controller  will  attempt  to  send  two  packets  back- 
to-back.  The  first  packet  is  sent  because  the  acknowledgment  that  just  arrived 
decreased  the  number  of  unacknowledged  packets  and  therefore  a  new  packet 
can  be  sent.  The  second  packet  is  sent  because  the  window  size  just  increased, 
allowing  the  controller  to  have  an  extra  unacknowledged  packet.  However,  at 
this  point  there  is  a  very  fragile  balance  between  the  number  of  packets  that 
are  getting  in  and  out  of  the  queue,  so  two  packets  will  not  fit  in  the  queue 
and  the  second  packet  is  dropped.  This,  of  course,  assumes  a  drop- tail  queuing 
policy.  Although  this  behavior  is  essentially  caused  by  the  discreteness  of  the 
queue  mechanism,  we  can  incorporate  it  in  our  hybrid  model  by  considering  two 
modes  for  the  system:  One  mode  corresponds  to  the  situation  when  the  queue 
is  not  full  and  therefore  the  system  evolves  according  to  (2),  (3),  (4).  The  other 
mode  of  operation  corresponds  to  the  situation  where  the  queue  is  full  and  one 
drop  will  occur  in  each  flow.  This  mode  of  operation  is  active  for  RTT  seconds. 
When  the  system  leaves  this  mode  all  window  sizes  are  multiplied  by  m  because 
of  the  multiplicative  decrease  caused  by  the  drops.  In  reality,  the  multiplicative 
decrease  of  all  flows  does  not  occur  exactly  at  the  same  time  instant.  However, 
this  model  provides  a  very  good  approximation  for  the  time  scales  considered 
here. 

Figure  2  contains  a  graphical  representation  of  the  overall  hybrid  system.  In 
this  figure,  each  node  represents  one  of  the  two  discrete  states:  queue-fall  and 
queue-not-full  The  continuous  state  of  the  hybrid  system  consists  of  the  queue 
size  q,  the  window  sizes  Wi,  i  e  {1,2,  •  ,n},  and  a  timing  variable  tr  used  to 

enforce  that  the  system  remains  in  the  queue-full  state  for  RTT  seconds.  The 
differential  equations  for  these  variables  in  each  discrete  state  are  shown  inside 
the  corresponding  nodes.  The  links  in  the  figure  represent  discrete  transitions, 
which  are  labeled  with  their  enabling  conditions  and  any  necessary  reset  of  the 
continuous  state  that  must  take  place  when  the  transition  occurs.  We  assume 
here  that  a  jump  always  occurs  when  the  transition  condition  is  enabled.  This 
model  falls  in  several  of  the  general  hybrid  systems  frameworks  proposed  in  the 
literature  [14,15,16,17,18,19,20,21,22].  For  simplicity  we  assume  here  that  the 
queue  size  q  never  reaches  zero. 

Remark  1.  For  a  very  large  number  of  flows,  a  single  drop  per  flow  may  not  be 
sufficient  to  produce  the  decrease  in  the  window  size  required  to  make  the  queue 
size  drop  below  q^ax  after  the  multiplicative  decrease.  In  this  case,  the  model  in 
Figure  2  is  not  valid.  However,  we  shall  see  in  Section  4  that,  for  most  operating 
conditions,  this  model  accurately  matches  packet-level  simulations  performed 
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Fig.  2.  Hybrid  model  for  Reno  congestion  control 


using  the  ns  2  network  simulator  [23].  In  fact,  this  hybrid  model  only  fails  when 
the  number  of  flows  is  so  large  that  the  drop  rates  take  unusually  large  values. 


3  Dynamics  in  Normalized  Time 

The  dynamics  for  the  hybrid  system  in  Figure  2  are  nonlinear  essentially  because 
of  the  dependence  of  RTT  on  q.  However,  it  is  possible  to  make  them  linear  by 
normalizing  the  time  variable.  To  this  effect  we  introduce  a  new  time  variable  r, 
called  the  normalized  defined  by 

—  =  RTT  =  Tp  +  ^,  t(0)  =  0.  (5) 

This  means  that  an  interval  with  duration  dr  in  the  variable  t  corresponds  to 
an  interval  of  duration  dt  =  RTTdr  in  the  variable  t.  We  can  think  of  r  as  a 
time  variable  normalized  so  that  one  unit  of  t  corresponds  to  one  round-trip 
time.  Figure  3  shows  the  dynamics  of  the  hybrid  system  in  normalized  time. 
In  this  figure,  '  denotes  the  derivative  ^  with  respect  to  the  normalized  time 
r.  In  Figure  3,  we  also  used  the  fact  that  in  the  queue-full  state,  q  =  qmax 
and  therefore,  waiting  until  tr  reaches  Tp  -f-  from  zero  with  =  RTT  = 
Tp  -h  2^^  is  equivalent  to  waiting  until  tt  reaches  1  from  zero  with  =  1. 

^  Formally,  there  is  a  bijective  function  /  that  maps  normalized  time  r  into  real  time 
t.  This  function  is  actually  defined  by  (5).  With  some  abuse  of  notation,  when  we 
write  q{T)  for  some  normalized  time  r,  we  really  mean  q{f{T)).  Similar  notation  is 
used  for  the  remaining  time-dependent  variables. 
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Fig.  3.  Hybrid  model  for  Reno  congestion  control  in  normalized  time. 


It  is  interesting  to  note  that  the  equation  that  models  the  queue  dynamics 
in  the  queue~not-full  state  is  stable.  This  is  an  important  property  of  window- 
based  congestion  control,  as  opposed  to  other  congestion  control  mechanisms 
that  adapt  the  packets  sending  rates  directly  (instead  of  indirectly  through  the 
window  size). 

Let  us  denote  by  {rk  :  <  Tk+ijk  >1}  the  set  of  normalized  times  at 

which  the  system  leaves  the  queue-full  mode.  Using  the  fact  that  the  system 
dynamics  is  essentially  linear  at  each  discrete  mode,  it  is  somewhat  tedious  but 
nevertheless  straightforward  to  show  that 

n+i  -Tk  =  f~^{sk)  +  1,  fc  >  1,  (6) 


where 

?max  +  BTp  — 

Sk  ’=  - ,  UJ 

an 

and  /  :  [0,  oo)  — )■  [0,  oo)  denotes  the  smooth  bijection 

|0  X  =  0 

The  reader  is  referred  to  [24]  for  the  detailed  derivation  of  (6).  We  proceed  to 
analyze  the  evolution  of  the  Wi{rk).  To  this  effect,  suppose  that  the  system  left 
the  queue- full  mode  at  some  normalized  time  Tk,  k  >  1.  Since  it  takes  f~^(sk)-hl 
units  of  normalized  time  until  the  system  leaves  the  queue-full  state  again  and 
during  this  time  =  a,  i  €  {1, 2, , . , ,  n},  we  conclude  that 

w~{Tk+i)  =Wi{Tk)-\-af~'^(sk)+a,  i  e  {l,2,...,n}, 
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and  therefore 

■WiiTk+i)  =m{wi{Tk) +  af-^(sk) +  a)  ,  i  e  {1,2, . . .  ,n}.  (8) 

From  (7)  and  (8)  we  then  conclude  that 

Sfc+I  =  m{sk  -  f~\sk))  +  ^-^(9max  +  BTp)  -  m.  (9) 

It  turns  out  that,  as  long  as 


,  ^  2ma 

Qmax  H“  BTp  >  - - 71, 

1  —  m 

the  map  g  :  [0,  oo)  — )>  [0,  oo),  defined  by 

s  •->  m{s  -  f-\s))  +  +  BTp)  -  m, 

is  a  contraction.  In  particular, 

\9{s)~9{s)\  =  m\s  ~  s  ~  f~\s)  +  f~\s,)\  <m|s-s|,  s,s>0.  (10) 

Since  (9)  can  also  be  written  as  Sk+i  =  g{sk),  using  the  Contraction  Mapping 
Theorem  [25,  p.  126]  we  conclude  that  the  Sk  converges  to  the  unique  fixed  point 
Soo  of  p,  which  is  the  unique  solution  to 

Soo  =  m(soo  -  /“H^oo))  +  +  BTp)  -  m.  (11) 

The  convergence  is  as  fast  as  m^.  Prom  this  and  (8)  we  conclude  that  the  fol- 
lowing  theorem  holds: 

Theorem  1.  Let  {tf^  :  tk  <  >  1}  be  the  set  of  times  at  which  the  system 

leaves  the  queue-full  For  +  BTp  >  ^n,  all  the  Wi{tk),  i  e  {1,2, ,  n} 
converge  exponentially  fast  to 

TirtCL  /  -  1  >  s. 

(^oo)  +  1), 

as  k  ^  oo  and  the  convergence  is  as  fast  as  m^. 

The  condition  q-max  +  BTp  >  essentially  limits  the  maximum  number  of 

flows  under  which  the  one- drop- per-flow  is  valid.  When  this  condition  is  violated, 
i.e.,  when  n  >  ^:^{qjna.x  +  BTp),  a  single  drop  per  flow  may  not  be  sufficient 
to  produce  a  decrease  in  the  sending  rates  that  would  make  q  drop  below  q^^^ 
after  the  multiplicative  decrease. 

A  straightforward  conclusion  of  Theorem  1  is  that  all  the  flows  become  syn¬ 
chronized  as  time  goes  to  infinity.  This  is  because  the  window  sizes  of  all  the 
flows  asymptotically  converge  to  the  same  limit  cycle.  This  limit  cycle  corre¬ 
sponds  to  an  increase  of  the  window  size  from  to  ~w^,  lasting  /“^(Sqo)  +  1 
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units  of  normalized  time,  followed  by  an  instantaneous  decrease  back  to  w^o  due 
to  drops. 

Window  size  synchronization  had  been  observed  in  [11]  for  Tahoe  congestion 
control  [4].  In  [11],  the  authors  defend  that  synchronization  is  closely  related  to 
the  packet  loss  synchronization  that  we  also  use  in  our  model.  In  fact,  they  pro¬ 
vide  an  informal  explanation — supported  by  packet-level  simulations — of  how 
synchronization  is  a  self-sustained  phenomenon.  Although  [11]  only  deals  with 
Tahoe,  the  arguments  used  there  also  apply  to  Reno  congestion  control.  Theo¬ 
rem  1  goes  further  because  it  demonstrates  that  the  limit  cycle  that  corresponds 
to  flow  synchronization  is  globally  exponential  stable.  This  means  that  synchro¬ 
nization  will  occur  even  if  the  flows  start  unsynchronized  or  lose  synchronization 
because  of  some  temporary  disturbance.  Moreover,  the  convergence  to  the  limit 
cycle  is  very  fast  and  is  reduced  by  at  least  m  (typically  1/2)  on  each  cycle.  In 
fact,  initially  the  convergence  is  even  faster  because  the  upper  bound  in  (10)  is 
conservative  for  large  |s  —  s|. 


4  Steady-State  Behavior 


We  proceed  now  to  derive  steady-state  formulas — such  as  the  ones  found  in 
[6, 7,8, 9] — that  relate  the  average  throughput,  the  average  drop  rate  (i.e.,  the 
percentage  of  dropped  packets),  and  the  average  round-trip  time.  In  this  section 
we  concentrate  on  the  case  where  Soo  is  much  larger  than  one  and  therefore 

/"H^oo)  ~  Soo  +  1.  (12) 


This  approximation  is  valid  when 


Qm&x  "b  BTp  ^ 


2man 
1  —  m 


(13) 


and  results  in  the  system  remaining  in  the  state  queue-not-full  for,  at  least,  a 
few  round-trip  times^.  In  practice,  this  is  quite  common  and  a  deviation  from 
(13)  results  in  very  large  drop  rates. 

Suppose  then  that  the  steady-state  has  been  reached  and  let  us  consider  an 
interval  [tk,tk+i\  between  two  consecutive  time  instants  at  which  the  system 
enters  the  queue-not-full  state.  Somewhere  in  this  interval  lies  the  time  instant 
4  at  which  the  system  enters  the  queue-full  state  and  drops  occur.  During  the 
interval  Ihe  instantaneous  rate  r  at  which  the  nodes  are  successfully 

transmitting  packets  is  given  by 


r{t)  =  I 


^  ^  [^fc?  ^/c) 

^  ^  [I'k^  ^fc+l] 


(14) 


^  When  the  system  remains  in  the  queue-not-full  for  at  least  4  round-trip  times,  (12) 
already  yields  an  error  smaller  than  2%. 
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The  total  number  of  packets  Nk  sent  during  the  interval  [4,4+1]  can  then  be 
computed  by 

r^k+i  r'Tk+i 

Nk  :=  /  r{t)dt  =  /  r{T)RTT{T)dT  (15) 

Jtk  Jrk 

We  used  here  the  change  of  integration  variable  defined  by  (5)  to  work  with 
quantities  in  normalized  time.  Details  on  the  computation  of  the  integrals  in 
(15)  and  in  (17)  below  are  given  in  [24].  Since  n  drops  occur  in  the  interval 
[4,4+i],  the  average  drop  rate  p  is  then  equal  to 


n  ^  2a  /  n 

Nk  1  -  m2  \^niax  H-  BTp  +  2an 


(16) 


Another  quantity  of  interest  is  the  average  round-trip  time  RTT.  We  consider 
here  a  packet-average,  rather  than  a  time-average,  because  the  former  is  the 
one  usually  measured  in  real  networks.  This  distinction  is  important  since  the 
sending  rate  r  is  not  constant.  In  fact,  when  the  sending  rate  is  higher,  the 
queue  is  more  likely  to  be  full  and  the  round-trip  time  is  larger.  This  results 
in  the  packet- average  being  larger  than  the  time-average.  The  packet-average 
round-trip  time  can  then  be  computed  as 


pw  ^mTT{t)dt  _  r{T)RTT{rYdr 

Nk  Nk 

_  1  /21-m3  +  l-m\ 

T\3  1-m'^  n  ^iTm J  ’ 


where  T  ^  is  the  average  throughput  of  each  flow.  We  recall  that,  because 
the  queue  never  empties,  the  total  throughput  is  precisely  the  bandwidth  B  of 
the  bottleneck  link. 


It  is  interesting  to  note  that  the  average  drop  rate  p  can  provide  an  estimate 
for  the  quantity  g^^^^BTp+2an'  particular,  we  conclude  from  (16)  that 


Q'max  “b  BTp  -|-  2an  ^  I  2a 

n  Y  (1  —  m‘^)p' 


(18) 


This,  in  turn,  can  be  used  together  with  (17)  to  estimate  the  average  throughput 
T.  In  fact,  from  (17)  and  (18)  we  conclude  that 


rp  ^  1  j  2a  l-m\ 

wt  (3 1  -  m2  y  (1  -  w2)p  “  “rr^  j 

For  a  =  1  and  m  =  1/2,  (19)  becomes  T  For  reasonable 

drop  rates,  the  term  dominates  over  1/3  and  (19)  matches  closely  similar 
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formulas  derived  in  [6, 7, 8, 9].  It  should  be  emphasized  that  the  derivations  in 
these  references  do  not  take  queuing  into  account  nor  its  effect  in  the  varia¬ 
tion  of  the  round-trip  time.  The  coupling  between  the  n  competing  flows  is  also 
ignored  and  therefore  no  theoretically-supported  claim  is  made  to  the  extent 
that  the  steady-state  solution  is  actually  reached  in  an  asymptotic  sense.  The 
hybrid  model  introduced  here  also  leads  to  a  more  complete  description  of  the 
steady-state  behavior  of  TCP  through  the  explicit  formulas  (16)  and  (17)  for  the 
average  round-trip  time  RTT  and  the  drop  rate  p  as  a  function  of  the  number 
of  flows  n.  It  is  important  to  emphasize  that  RTT  in  (17)  denotes  the  aver¬ 
age  round-trip  time.  It  turns  out  that  the  actual  round-trip-time  RTT  varies 
quite  significantly  around  this  average  because  of  fluctuations  on  the  queue  size. 
These  large  variations  in  the  queue  size  (which  are  amplified  by  synchronization) 
produce  a  large  delay  jitter.  These  phenomena,  which  have  significant  implica¬ 
tions  in  the  design  of  congestion  control  mechanisms  for  applications  that  require 
stricter  service  guarantees  from  the  network,  have  not  been  accurately  captured 
in  most  existing  models  [9,26,27,10,28]. 

To  verify  the  formulas  derived  above,  we  simulated  the  dumbbell  of  Figure  1 
using  the  ns-2  network  simulator  [23].  Figure  4  summarizes  the  results  obtained 

for  a  network  with  the  following  parameters:  B  =  g  bits/chlrx  U3?o^Slr/packet  = 
1250  packets/sec,  Tp  =  .04  sec,  q^ax  =  250  packets,  a  =  1  packet/RTT, 
m  =  1/2.  As  seen  in  the  Figure  4,  the  theoretical  predictions  given  by  (16), 
(17),  (19)  match  the  simulation  results  quite  accurately.  Some  mismatch  can 
be  observed  for  large  number  of  flows.  However,  this  mismatch  only  starts  to 
become  significant  when  the  drop  rates  are  around  1%,  which  is  an  unusually 
large  value.  This  mismatch  is  mainly  due  to  two  factors:  the  quantization  of  the 
window  size  and  a  crude  modeling  of  the  fast-recovery  algorithm  [2].  We  are 
now  in  the  process  of  incorporating  these  two  features  into  our  hybrid  model  to 
obtain  formulas  that  are  accurate  also  in  very  congested  networks. 


5  Conclusion 

In  this  paper  we  proposed  a  hybrid  model  for  Reno  congestion  control.  Using 
this  model,  we  analyzed  both  the  transient  and  the  steady-state  behavior  of  n 
TCP  flows  competing  for  the  available  bandwidth  on  a  dumbbell  network  topol¬ 
ogy.  Our  model  confirmed  formulas  for  the  steady-state  behavior  that  can  be 
found  in  the  literature  and  also  derive  new  relationships  between  the  several 
quantities  of  interest.  We  were  also  able  to  explain  the  flow  synchronization  phe¬ 
nomena  that  have  been  observed  in  simulations  and  in  real  networks  but,  to 
the  best  of  our  knowledge,  have  not  been  theoretically  justified.  We  were  also 
able  to  demonstrate  that  the  limit  cycle  that  corresponds  to  flow  synchroniza¬ 
tion  is  globally  exponential  stable.  This  means  that  synchronization  will  occur 
even  if  the  flows  start  unsynchronized  or  lose  synchronization  because  of  some 
temporary  disturbance. 
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Drop  rate  Average  round-trip  time 


Fig.  4.  Comparison  between  the  predictions  obtained  from  the  hybrid  model  and  the 
results  from  ns-2  simulations. 


We  are  now  in  the  process  of  generalizing  the  analysis  presented  here  to  dif¬ 
ferent  network  topologies;  other  congestion  control  mechanisms  (such  as  Tahoe, 
Vegas,  and  Equation-Based);  and  different  queuing  policies  (such  as  drop-head, 
Random  Drop,  RED,  and  SRED).  We  are  also  exploring  mechanisms  that  can 
be  used  to  avoid  the  undesirable  synchronization.  Another  application  of  the 
hybrid  model  derived  here  is  the  detection  of  abnormalities  in  TCP  traffic  flows. 
This  has  important  applications  in  network  security. 


Appendix 

To  derive  equation  (6),  suppose  that  drops  occurred  at  some  normalized  time 
at  which  the  system  entered  the  queue-not-full  state  and  therefore  that  q{rk)  = 
Qmax-  Denoting  by  the  normalized  time  at  which  the  next  drop  occurs,  for 
r  £  [rjt,  ffc),  we  have 

Wi{r)  =  Wi{Tk)  +  a(r  -  rjt),  ^  G  {1, 2, . . . ,  n}, 
gr(r)  =  +  BTp  -b  an  -  + 

i=l 

n 

+  an(r  -  t^)  +  -  BTp  -  an. 

i=l 
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We  assumed  here  that  q  remains  positive  during  the  whole  interval.  Since  a  new 
drop  occurs  at  the  normalized  time  fk^  we  must  have  q{fk)  =  ^max-  Because  of 
gf’s  continuity,  we  must  then  have 

n 

9max  =  (^max  +  BTp  +  an  -  ^  Wi(rfc)j  + 

i=l 

n 

+  an(fk  -  Tk)  +  -  BTp  ~  an, 

4=1 

We  can  then  solve  this  equation  to  compute  the  normalized  time  interval  fk  —  Tk 
and  obtain 


Qmax  ~1~  BTp  ^2=1  '^k _ ^ 

an  1  —  ’ 

which  is  equivalent  to  fk  —  T‘k  —  Equation  (6)  is  a  consequence  of 

this  and  the  fact  that  the  system  enters  the  queue-not-full  state  again  at  time 

Tk+i  Tfc  +  1.  □ 
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Abstract.  In  this  paper  we  address  the  problem  of  designing  energy 
minimizing  collision- free  maneuvers  for  multiple  agents  moving  on  a 
plane.  We  show  that  the  problem  is  equivalent  to  that  of  finding  the 
shortest  geodesic  in  a  certain  manifold  with  nonsmooth  boundary.  This 
allows  us  to  prove  that  the  optimal  maneuvers  are  by  introducing 
the  concept  of  u-convex  manifolds.  Moreover,  due  to  the  nature  of  the 
optimal  maneuvers,  the  problem  can  be  formulated  as  an  optimal  con¬ 
trol  problem  for  a  certain  hybrid  system  whose  discrete  states  consist 
of  different  “contact  graphs” .  We  determine  the  analytic  expression  for 
the  optimal  maneuvers  in  the  two  agents  case.  For  the  three  agents  case, 
we  derive  the  dynamics  of  the  optimal  maneuvers  within  each  discrete 
state.  This  together  with  the  fact  that  an  optimal  maneuver  is  a  con¬ 
catenation  of  segments  associated  with  different  discrete  states  gives  a 
characterization  of  the  optimal  solutions  in  the  three  agents  case. 


1  Introduction  and  Background 

Many  problems  arising  in  practical  situations  have  boundary  constraints  and  can 
be  described  in  the  setting  of  manifolds  with  boundary.  Here  we  are  interested 
in  certain  geometric  aspects  of  such  manifolds,  specifically  those  concerning  the 
properties  of  geodesics^  i.e.,  locally  distance  minimizing  curves.  It  is  intuitively 
clear  that  when  the  boundary  consists  of  cells  of  various  dimensions  pieced  to¬ 
gether,  a  geodesic  is  in  general  “hybrid”  in  the  sense  that  it  is  a  concatenation 
of  different  segments,  each  one  of  which  being  a  geodesic  of  a  particular  cell  (in 
its  own  geometry).  Thus  in  the  hybrid  systems  terminology  ([16])  ,  the  geodesics 
can  be  naturally  described  as  the  executions  of  an  underlying  hybrid  system. 
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Note  that  our  interpretation  of  a  manifold  with  boundary  as  the  domain  for 
the  continuous  state  of  a  hybrid  system  is  the  inverse  of  the  procedure  adopted 
in  [21],  where  the  concept  of  hybrifold  is  introduced  by  piecing  together  the  do¬ 
mains  corresponding  to  all  the  discrete  modes  of  a  hybrid  system  to  form  a  single 
topological  manifold.  Another  difference  is  that,  in  addition  to  the  topological 
properties  of  the  hybrid  systems  such  as  stability,  zenoness,  ergodicity,  etc.,  we 
are  also  interested  in  their  metric  properties  such  as  distance,  curve  length,  an¬ 
gle,  etc.  Therefore  when  piecing  domains  together,  isometries  instead  of  merely 
diffeomorphisms  are  required  as  the  identifying  maps  of  the  boundaries. 

To  be  precise,  let  M  be  a  connected  m-dimensional  Riemannian  man¬ 
ifold  with  boundary.  The  boundary  of  M  can  be  either  smooth  or  nonsmooth. 
Consider  only  those  curves  in  M  which  are  piecewise  i.e.,  curves  which  can 
be  partitioned  into  a  countable  number  of  segments.  For  such  curves  the  arc 
length  is  well  defined.  The  distance  between  two  points  in  M  is  then  defined  as 
the  infimum  of  the  arc  length  of  all  the  piecewise  curves  connecting  them.  A 
geodesic  in  M  is  a  locally  distance  minimizing  curve.  More  precisely,  the  curve 
7  :  (to,  ^/)  M  is  a  geodesic  if  and  only  if  for  each  t  G  (to,  ^/),  7  is  the  shortest 
curve  between  7(ti)  and  7(t2)  for  every  ti,  t2  belonging  to  a  neighborhood  of  t 
with  ti  <  t  <  t2.  Given  two  arbitrary  points  in  M,  the  (globally)  shortest  curve 
connecting  them  is  automatically  a  geodesic.  However,  it  is  well  known  that  the 
converse  is  not  true:  a  geodesic  is  not  necessarily  distance  minimizing  between 
its  end  points.  In  fact,  even  for  manifolds  without  boundary,  a  geodesic  is  no 
longer  distance  minimizing  after  its  first  conjugate  point  ([5]). 

Due  to  the  presence  of  the  boundary,  regularity  of  geodesics  in  M  is  an 
issue.  The  special  case  of  geodesics  in  manifolds  with  smooth  boundary  is  dealt 
with  in  [2,3],  to  name  a  few.  We  now  review  briefly  some  of  the  results  in  these 
papers  relevant  to  our  study  in  the  nonsmooth  boundary  case.  For  manifolds 
with  smooth  boundary,  it  is  shown  in  [3]  that  geodesics  are  in  general  but 
not  C^.  The  simplest  example  is  with  a  unit  disk  removed.  Two  points  across 
the  disk  and  “invisible”  to  each  other  are  connected  by  at  most  two  shortest 
geodesics,  which  are  everywhere  but  fail  to  be  (7^  at  exactly  the  points  where 
geodesics  switch  from  a  line  segment  to  a  boundary  arcs  or  vice  versa.  In  [3]  it  is 
further  suggested  that  a  geodesic  in  a  manifold  M  with  smooth  boundary  can  be 
decomposed  into:  (1)  interior  segments,  which  are  geodesic  segments  belonging  to 
the  interior  of  M;  (2)  boundary  segments,  which  are  geodesic  segments  belonging 
to  the  boundary  dM  of  M ;  (3)  switch  points,  which  are  points  where  the  geodesic 
switches  from  a  boundary  segment  to  an  interior  segment  or  vice  versa]  (4) 
intermittent  points,  which  are  accumulation  points  of  the  set  of  switch  points.  It  is 
proved  in  [1]  that  when  the  boundary  dM  is  locally  analytic,  a  geodesic  can  have 
only  a  finite  number  of  switch  points  in  any  segment  of  finite  arc  length,  hence 
no  intermittent  points  at  all.  In  our  interpretation  of  geodesics  as  the  executions 
of  an  underlying  hybrid  system,  switch  points  correspond  to  transitions  between 
discrete  states,  and  the  existence  of  intermittent  points  in  a  geodesic  implies  that 
the  corresponding  execution,  hence  the  hybrid  system,  is  Zeno  ([16]).  Therefore 
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the  result  in  [1]  can  be  rephrased  by  saying  that  a  hybrid  system  whose  executions 
correspond  to  geodesics  in  a  manifold  with  locally  analytic  boundary  is  non- Zeno. 

In  this  paper  we  study  the  problem  of  optimal  collision-free  motion  planning 
for  multiple  agents  moving  on  a  plane,  where  a  collision  is  the  event  that  any  two 
agents  get  closer  than  a  minimum  allowed  distance.  We  show  that  each  collision- 
free  joint  maneuver  has  a  natural  representation  as  a  curve  in  a  certain  manifold 
with  boundary,  and  among  all  such  joint  maneuvers  the  one  with  the  least  energy 
corresponds  to  a  geodesic  parameterized  proportionally  to  arc  length.  Geodesics 
satisfying  this  property  are  called  normalized.  Unless  otherwise  stated,  we  assume 
throughout  the  paper  that  all  geodesics  are  normalized. 

The  problem  which  inspired  this  work  originally  is  the  development  of  algo¬ 
rithms  for  aircraft  conflict  resolution.  Aircraft  flying  at  the  same  altitude  must 
maintain  a  horizontal  separation  of  at  least  3  nautical  miles  (nmi)  inside  the  ter¬ 
minal  radar  approach  control  facilities  and  5  nmi  in  the  en-route  airspace  ([20]). 
Moreover,  the  energy  of  an  aircraft  maneuver  is  closely  related  to  practical  as¬ 
pects  such  as  travel  distance,  fuel  consumption,  passenger  comfort,  etc.  Numer¬ 
ous  approaches  have  been  suggested  in  the  literature  to  deal  with  aircraft  conflict 
resolution  (see  the  survey  paper  [13]).  Some  of  them  ([6,8,11,17])  actually  pose 
the  problem  as  a  constrained  optimization  problem.  In  particular,  in  [11]  the 
geometric  interpretation  of  aircraft  motions  as  a  braid  is  used  in  performing  the 
optimality  analysis.  Optimal  multi- agent  coordination  also  finds  applications  in 
other  transportation  systems,  for  example  [18].  Another  related  field  is  the  mo¬ 
tion  planning  for  mobile  robots.  Most  of  the  papers  in  this  field  focus  on  the 
feasibility  and  the  algorithmic  complexity  aspect  of  the  problem  ([7,9,14,22]). 
Among  those  dealing  with  optimal  coordination,  [15]  considers  the  case  when 
each  robot  minimizes  its  own  independent  goal  by  using  techniques  from  multi¬ 
objective  optimization  and  game  theory.  [4]  studies  the  problem  of  time-optimal 
control  of  multiple  vehicles  moving  on  a  plane  with  constant  speed  and  bounded 
curvature. 

The  rest  of  the  paper  is  organized  as  follows.  In  Sect.  2,  we  describe  the  opti¬ 
mal  collision-free  motion  planning  problem  and  show  how  it  can  be  reformulated 
as  the  problem  of  finding  the  shortest  geodesic  in  a  manifold  M  with  nonsmooth 
boundary.  Using  the  fact  that  M  is  a  u-convex  manifold,  we  are  able  to  prove 
in  Sect.  2.2  that  the  optimal  motions  for  the  agents  are  C^.  We  then  introduce 
in  Sect.  3  the  notion  of  “contact  graph” ,  which  leads  to  a  natural  interpretation 
of  the  problem  in  the  framework  of  optimal  control  for  a  certain  hybrid  system. 
The  property  implies  that  the  reset  maps  of  the  hybrid  system  are  all  identity 
maps.  The  shortest  geodesic  can  be  obtained  by  appropriately  piecing  together 
geodesic  segments  in  different  discrete  modes,  and  is  the  optimal  execution  for 
the  hybrid  system.  In  Sect.  3.1  necessary  conditions  are  introduced  to  simplify 
the  determination  of  such  geodesics,  which  are  then  used  in  Sect.  3.2  and  3.3 
to  characterize  the  optimal  collision-free  motions  for  the  two  agents  and  three 
agents  case  respectively.  Finally  some  concluding  remarks  are  given  in  Sect.  4. 
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2  Problem  Formulation 

Consider  the  situation  when  n  agents,  numbered  from  1  to  n,  are  moving 
on  a  common  plane  The  n  agents  are  required  to  start  from  positions 
ai,...  ,  ttn  G  at  time  to  and  reach  positions  6i , . . .  ,  G  at  a  fixed  time 
tf.  We  assume  that  each  one  of  the  two  sets  and  {bi}2^i  satisfies  the  r- 

separation  condition  for  some  positive  r,  in  the  sense  that  the  minimum  pairwise 
Euclidean  distance  in  each  set  is  at  least  r. 

A  maneuver  for  agent  1  <  z  <  n,  is  defined  to  be  a  piecewise  map 
•  [^05^/]  ->  satisfying  ai(to)  =  cii  and  ai{tf)  =  6^.  The  set  of  all  maneuvers 
for  agent  i  is  denoted  as  Vi.  Then  V  —  nil  Vi  is  the  set  of  joint  maneuvers  for 
the  n-agent  system.  Here  we  are  interested  in  the  subset  V{r)  of  V  consisting  of 
all  the  collision-free  maneuvers.^  i.e.,  those  joint  maneuvers  a  =  (ai, . . ,  ,  a„)  G  V 
such  that  satisfies  the  r-separation  condition  at  each  time  t,  t  G  [to,  i/]- 

The  energy  of  a  joint  maneuver  a  =  (ai, . . .  ,  G  P  is  defined  as 

^  i=l 

The  goal  is  to  find  the  collision-free  maneuver  a  G  V{r)  with  minimal  energy. 
This  leads  to  the  following  formulation  of  the  problem: 

Minimize  J(a)  subject  to  a  G  V{r).  (1) 

Notice  that  in  formulating  problem  (1),  we  make  the  restrictive  assumption 
that  all  the  agents  involved  in  the  encounter  reach  their  destinations  at  the  same 
known  time  instant  tf.  This  is  important  in  time-critical  applications  such  as  air 
traffic  management.  The  issue  of  choosing  tf  is  not  dealt  with  in  this  paper. 

Remark  1.  Problem  (1)  can  be  alternatively  formulated  as  an  optimal  control 
problem  with  state  constraints,  and  approached  by  using  the  corresponding  tech¬ 
niques  from  optimal  control  theory.  In  this  paper,  however,  we  adopt  a  geometric 
point  of  view.  The  geometric  method  not  only  yields  elegant  results  and  proofs, 
but  more  importantly,  by  using  information  on  the  curvature  of  the  domains,  it 
also  allows  us  to  obtain  deeper  results  concerning  the  global  uniqueness  of  the 
optimal  solutions  under  certain  conditions  (see  [12]). 

2.1  A  Geometric  Interpretation 

Each  joint  maneuver  a  =  (oi,...  ,an)  in  V  can  be  re-interpreted  as  a  curve 
in  defined  by  a{t)  =  (ai(t), . . .  ,an(t)),  t  G  [to,fy],  which  starts  from  a  = 
(ai, . . .  ,  a„)  and  ends  at  6  =  (6i, . . .  ,  6^).  If  we  use  (xi,  yi, . . .  ,  y„)  to  denote 

the  coordinates  of  a  generic  point  in  then  the  collision-free  constraint  on 
the  joint  maneuver  a  translates  into  the  condition  that  a  viewed  as  a  curve  in 
is  strictly  contained  in  M,  a  manifold  with  boundary  obtained  by  removing 
from  the  “static  obstacle”  W  given  by 


^  =  {P  €  +  {Vi  ~  VjY  <  r  for  some  I  <i  <  j  <n}.  (2) 
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In  other  words,  M  —  Thus  there  is  a  one-to-one  correspondence  between 

maneuvers  in  7^(7’)  and  piecewise  curves  in  M  connecting  a  and  b.  Moreover, 
the  energy  of  a  joint  maneuver  a  =  (cki,...  ^an)  ^  V  can  be  expressed  as 
J(a)  =  I  Ito  dt  =  ^  ||d(t)p  dt,  which  coincides  with  the  usual 

definition  of  the  energy  of  a  viewed  as  a  curve  in  Hence  (1)  is  equivalent 
to  the  following  geometric  problem: 

Find  the  energy  minimizing  curve  a  in  M  joining  point  a  to  point  b.  (3) 

It  is  a  standard  result  (see,  e.g.,  [19])  that  solutions  to  (3)  are  shortest  curves 
in  M  from  a  to  b  parameterized  proportionally  to  arc  length,  i.e.,  minimizing 
geodesics  in  M  connecting  a  to  6.  We  shall  henceforth  study  problem  (3)  instead 
of  (1)  with  the  understanding  that  all  the  curves  connecting  a  to  6  in  M  are 
parameterized  so  that  they  start  from  a  at  to  and  end  at  6  at  t/. 

Notice  that  W  defined  in  (2)  is  the  union  of  n(n  — 1)/2  convex  open  cylinders, 
each  one  of  the  form  ,Xn,yn)  '  \/{^i  -  +  {Vi  ~  VjY  <  f'}  for 

some  (i,  j),  with  i  ^  j.  Therefore  M  obtained  by  removing  W  from  is  an 
instance  of  the  following  class  of  manifolds  with  boundary: 

Definition  1  (u-convex  manifolds).  A  manifold  with  boundary  is  called  u- 
convex  if  it  is  obtained  by  removing  from  some  Euclidean  space  a  finite  union 
of  open  convex  subsets,  each  one  of  which  has  a  smooth  boundary. 

We  will  show  in  the  next  section  that  geodesics  in  u-convex  manifolds  are 

which  implies  that  solutions  to  problem  (3),  hence  (1),  are  C^. 

2.2  Geodesics  in  u- Convex  Manifolds 

In  this  section  we  study  the  properties  of  geodesics  in  u-convex  manifolds.  Many 
technicalities  encountered  in  the  general  case  can  be  avoided  when  analyzing  this 
special  case.  For  example,  when  the  boundary  of  M  is  nonsmooth,  geodesics  in 
M  are  in  general  not  since  they  can  bend  into  sharp  corners  of  the  boundary. 
However,  we  next  show  that  this  is  not  the  case  for  u-convex  manifolds. 

Suppose  M  is  u-convex,  i.e.,  M  =  R"^  \  is  the  complement  in  R^ 

of  the  union  of  open  convex  bodies  Di,. . .  ,Dk  C  R”^,  whose  boundary  dDi  is 
smooth  for  each  i  =  1, . . .  ,k.  Then  at  each  point  x  G  M,  we  can  define  the  visible 
cone  of  X  to  be  the  cone  V (x)  with  vertex  x  and  consisting  of  all  the  rays  which 
start  from  x  and  lie  inside  M  within  a  sufficiently  small  distance.  In  other  words, 
V  (x)  is  the  region  a  viewer  sitting  at  x  can  see  if  only  local  obstacles  around  x 
are  considered.  V{x)  can  be  obtained  in  the  following  way  If  x  G  M  lies  on  the 
boundary  of  Di  for  exactly  those  i  belonging  to  a  subset  X  of  {1, . . . ,  A:},  then  the 
obstacles  Di,  i  are  called  the  active  obstacles  at  x.  For  each  active  obstacle 
Di,  let  Tx{dDi)  be  the  plane  tangent  to  dDi  at  x  and  be  the  unit  normal 
vector  of  dDi  at  x  pointing  outside  of  Di.  Tx{dDi)  separates  R^  into  two  open 
half  spaces.  We  denote  the  one  containing  as  and  its  closure  as  K.-  The 
convexity  of  Di  implies  that  and  Di  are  disjoint  sets.  Then  V (x)  is  given  by 
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Fig.  1.  Examples  of  visible  cones.  On  the  right  a  degenerate  case. 


V{x)  =  fliei  is  a  closed  convex  cone  since  it  is  the  finite  intersection 

of  closed  convex  sets  (half  spaces) ,  and  it  can  have  an  arbitrary  dimension  lower 
than  m.  Figure  1  shows  examples  of  visible  cones  in  In  the  case  when  x  is 
in  the  interior  of  M,  V (x)  =  since  there  are  no  active  obstacles  at  x. 

By  using  the  notion  of  visible  cone,  one  can  prove  the  following  result. 

Theorem  1.  Suppose  that  M  is  u-convex.  Then  any  geodesic  in  M  is  . 

Proof.  Let  7  :  /  — )•  M  be  a  geodesic  of  M,  where  I  is  an  open  interval  in  M.  For 
each  s  G  /,  the  one-sided  derivatives  'y'(s~)  and  7'(s+)  of  7  at  s  exist  since  7 
is  piecewise  .  By  using  a  reparameterization  if  necessary,  we  can  assume  that 
both  of  them  are  unit  vectors.  Construct  the  visible  cone  V{x)  of  x  =  7(5).  By 
definition,  both  7'(5'*’)  and  — 7'(s~)  based  at  x  lie  inside  V(x)  and  they  span  an 
angle  9  e  [0,  tt].  Suppose  by  contradiction  that  7'(s“)  ^  7'(s'*'),  then  9  <  tv. 

Fix  a  neighborhood  ^7  of  x  small  enough  so  that  only  the  active  obstacles 
at  X  intersect  U.  Choose  e  such  that  7|[5_g5+e]  C  U.  For  each  t  G  [s  —  6,s], 
let  7(i)  be  the  projection  of  7(t)  onto  the  line  through  x  and  along  the  direc¬ 
tion  —7^(5“);  for  each  t  G  [s,  s  +  e],  let  7(t)  be  the  projection  of  7(t)  onto  the 
line  through  x  and  along  the  direction  7'(s‘^).  Notice  that  7|[5_€,s-|_e]  is  a  curve 
through  X  contained  completely  within  M.  By  choosing  e  small  enough,  one 
can  ensure  that  the  line  segments  7(5  -  6)7(5  —  e)  and  7(5  +  5)7(5  -f  e)  both  lie 
completely  inside  M.  Therefore  by  replacing  the  arc  7|[5_g^5+g]  with  the  con¬ 
catenation  of  7(5  -  6)7(5  -  6),  the  arc  7|[s_g^5+e],  and  7(5  +  6)7(5  -h  6),  the  to¬ 
tal  arc  length  is  increased  by  at  most  0(6).  Notice  further  that  we  can  short¬ 
cut  7|[5_g^5+e]  by  the  line  segment  7(5  —  6)7(5  -1-  6),  which  lies  completely  inside 
V (x)  (hence  M)  by  the  convexity  of  V (x).  Doing  so  can  reduce  the  arc  length  of 
7|[s_£,s+e]  by  at  least  26(1  —  sin(0/2))  -{-  0(5),  where  we  use  the  fact  that  7'(s”) 
and  7^(s~*~)  are  unit  vectors.  Therefore  the  concatenation  of  the  line  segments 
7(5  —  6)7(5  —  6),  7(5  —  6)7(5  +  6),  and  7(5  +  6)7(5  +  6)  is  a  curve  in  M  shorter 
than  the  arc  7|[s_e^5_l_e]  for  6  small  enough.  This  contradicts  the  fact  that  7  is 
locally  distance  minimizing.  Thus  9  =  tv  and  7  is  ever3rwhere. 

To  show  the  necessity  of  u-convexity  in  proving  Theorem  1,  we  plot  in  Fig.  2 
an  example  in  which  M  is  obtained  by  removing  from  a  nonconvex  obstacle 
given  by  the  exterior  Di  of  a  cylinder  with  axis  li  and  a  convex  obstacle  given  by 
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Fig.  2.  Geodesic  in  a  manifold  with  boundary  that  is  not  u-convex. 


the  interior  D2  of  a  cylinder  with  the  same  radius  and  with  axis  I2  intersecting 
at  a  right  angle.  Hence  M  consists  of  all  those  points  in  which  lie  inside 
the  cylinder  with  axis  li  but  outside  the  cylinder  with  axis  ^2,  with  the  points 
on  their  boundaries  included.  The  heavy- weighted  curve  in  Fig.  2  is  a  geodesic 
in  M  with  end  points  a  and  6,  which  is  clearly  not  at  rc. 

3  Hybrid  System  Solution 

Now  we  go  back  to  the  discussion  of  the  optimization  problem  (3)  proposed  in 
Sect.  2,  where  M  =  \  W  with  W  defined  in  (2). 

Consider  a  curve  a  =  (ai, . . .  ,  a„)  from  a  to  6  in  M  corresponding  a  collision- 
free  maneuver  in  'P(r).  Fix  a  time  instant  t  6  [to,tf].  We  say  that  agent  i  and 
agent  j  contact  at  time  t  if  and  only  if  ||o;i(t)  —  (yj{t)\\  —  r.  A  graph  can  be 
associated  to  a  at  time  t  in  the  following  way.  The  graph  has  n  vertices,  numbered 
from  1  to  n,  each  one  corresponding  to  an  agent,  and  an  edge  exists  between 
vertex  i  and  vertex  j  if  and  only  if  agent  i  and  agent  j  contact  at  time  t.  We 
call  this  graph  the  contact  graph  of  a  at  time  t. 

Let  a*  be  a  curve  from  a  to  6  in  M  that  is  a  solution  to  problem  (3).  Suppose 
that  there  is  a  finite  subdivision  of  [to,tf]:  to  <  ti  <  ...  <  tk-i  <tk  =  tf,  such 
that  the  contact  graph  of  a*  over  the  suhinterval  {th^i^th)  (which  we  denote  as 
Gh)  remains  constant  for  all  h  =  1,...  ,  fc,  while  contiguous  subintervals  have 
distinct  contact  graphs.  In  each  subinterval,  say  {th~i,th)^  a*  moves  on  a  certain 
part  of  M  determined  by  If  Gh  has  no  edges,  then  a*  restricted  to  {th~i,th) 
is  a  straight  line  segment  in  the  interior  of  M.  If  Gh  has  at  least  one  edge, 
then  a*  restricted  to  (th-i^th)  moves  on  a  portion  of  the  boundary  of  M,  which 
is  a  lower  dimensional  smooth  submanifold  of  consisting  of  all  the  points 
•  •  •  j^n^Vn)  in  such  that  y/ {xi  —  Xj)‘^  +  (pi  —  yj)'^  is  equal  to  r  for 
(i,  j)  such  that  there  is  an  edge  between  vertices  i  and  j  in  G/^,  and  greater  than 
r  for  all  others  (i,  j),  i  j.  Moreover,  a*  restricted  to  (th-i^th)  is  a  minimizing 
geodesic  in  this  submanifold.  In  this  way  we  can  associate  to  each  type  of  contact 
graph  a  domain,  i.e.,  the  submanifold  of  M  to  which  a*  belongs  when  its  contact 
graph  is  of  that  type. 
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Based  on  the  above  analysis,  a*  can  be  viewed  as  an  execution  of  a  certain 
hybrid  system,  whose  continuous  variable  takes  values  in  M,  and  whose  discrete 
modes  have  a  one-to-one  correspondence  with  the  different  contact  graphs  for 
the  n- agent  system.  For  each  discrete  mode,  the  invariant  set  is  the  domain  of 
the  corresponding  contact  graph,  and  the  dynamics  is  governed  by  the  geodesic 
equation  on  that  domain,  which  is  a  second-order  ordinary  differential  equation. 
By  Theorem  1,  when  a  transition  occurs  between  discrete  modes,  the  position  a 
and  the  velocity  d  are  reset  by  identity  maps,  a  is  an  optimal  solution  to  this 
hybrid  system  if  it  satisfies  Q:(to)  =  a  and  a{tf)  =  6,  and  has  minimal  energy. 
The  problem  is  to  determine  the  initial  velocity  d(to)  and  the  time  and  sequence 
of  the  discrete  switchings  so  that  the  corresponding  execution  of  this  hybrid 
system  will  generate  the  optimal  solution. 

3.1  Necessary  Conditions  for  Optimality 

We  now  derive  some  necessary  conditions  for  a  to  be  an  optimal  solution  to 
problem  (3),  which  can  then  be  used  to  simplify  the  determination  of  optimal 
maneuvers  for  the  two- agent  and  three- agent  cases. 

Proposition  1.  Suppose  that  a*  is  a  minimizing  geodesic  from  a  to  b  in  M . 
Fix  an  arbitrary  u;  e  M^.  Then  /3*  =  (/?*,.. .  ,/5*)  defined  by 

Pi  (.t)  =  Q*  (0  +  w  i  =  (4) 

is  a  minimizing  geodesic  from  a  to  b'  =  {bi  w, . . .  ,  6^  +  u?)  in  M. 

Proof  For  each  curve  j3  from  a  to  6'  in  M,  define  curve  a  =  (oi, . . .  ,Q!„)  = 
T-w{P)  in  as  ai{t)  =  pi{t)  -  for  t  e  [to,tf]  and  i  =  1, . . .  ,n.  Then 

it  is  easily  verified  that  a  is  a  curve  in  M  from  a  to  6  with  energy 

J{a)  =  m  +  .  (5) 

tf  -  to 

The  second  term  of  the  right  hand  side  of  (5)  is  a  constant  independent  of 
which  we  shall  denote  as  C.  From  (5)  and  the  optimality  of  a*,  we  have 
J{fi)  ~  J{(y)  —  C  >  J{<y*)  ~  C  —  where  the  last  equality  follows  by 

noticing  that  a*  =  T-w{fi*)-  This  is  true  for  arbitrary  /?,  hence  the  conclusion. 

One  important  implication  of  Proposition  1  is  that  it  suffices  to  solve  prob¬ 
lem  (3)  only  for  those  a  and  b  that  are  aligned,  i.e.,  a  and  b  with  the  same 
centroid  ~  ^  Sr=i  ^  In  fact  for  non-aligned  a  and  b,  by  choos¬ 
ing  w  ~  ensure  that  a  and  F  =  {bi+w,...  ,bnPw) 

are  aligned.  Hence  by  Proposition  1,  minimizing  geodesics  from  a  to  6  can  be 
obtained  from  minimizing  geodesics  from  a  to  b'  by  applying  the  inverse  of  the 
transformation  (4). 
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Proposition  2.  Assume  that  a*  is  a  minimizing  geodesic  from  a  to  b  in  M. 
Then 


(</  -  t)  E?=1  aj  +  (*  -  ^o)  Iir=i 

tf  —  to 


^te[to,tf]. 


Proof.  Consider  first  the  case  when  a  and  b  are  aligned.  Define  a  piecewise 
map  g  :  [to,tf]  ^  t  e  [to,t/],  which  satisfies 

g(to)  =  g{tf)  =  0.  For  each  A  €  E,  let  /?a  =  ^/^A.n)  be  given  by 

^x,ilt)  =  af{t)  +  A g{t),  t  e  [to,tf],  i  -  1,  •  •  ■  ,  n.  Note  that  /?a  is  a  piecewise 
curve  from  a  to  6  in  M  with  energy 


^  dto  i— 1  •'*0  i—i 


dt. 


The  difference  J{^x)  —  J{a*)  is  a  quadratic  function  of  A  which,  by  the  optimality 
of  a*,  must  be  nonnegative  for  all  A.  Hence  we  have  ||  Y17=i  =  bj 

implying  that  ^  ^or  almost  all  t  e  [to,tf].  After  integration,  this 

leads  to  the  desired  conclusion  for  the  aligned  case.  In  the  case  when  a  and  b  are 
not  aligned,  the  result  follows  from  Proposition  1  by  considering  a  minimizing 
geodesic  in  M  from  a  to  b'  =  {bi  . . .  ,bn  +  w)  with  w  =  bi 

and  noticing  that  a  and  b'  are  aligned. 

A  geometric  interpretation  of  the  above  results  is  as  follows.  Let  N  be 
the  two  dimensional  subspace  of  E^”^  spanned  by  vectors  (0,1,...  ,0,1)  and 
(1,0,...  ,1,0),  and  V  be  the  orthogonal  complement  of  N  in  E^’^  such  that 
E^n  =  0  jv.  Then  the  condition  that  a  and  b  are  aligned  is  equivalent  to  the 

condition  that  b  -  a  belongs  to  V.  Denote  with  Va  the  (n  -  2)-plane  through  a 
and  parallel  to  V.  Then  if  a  and  b  are  aligned,  they  both  belong  to  14,  and  by 
Proposition  2,  a  minimizing  geodesic  a*  in  M  from  a  to  6  lies  in  Va  completely. 
For  non-aligned  a  and  6,  let  b'  be  the  orthogonal  projection  of  6  onto  14 .  Then 
Proposition  1  and  Proposition  2  say  that  a  minimizing  geodesic  a*  between  a 
and  bin  M  can  be  decomposed  into  two  parts:  its  projection  onto  14,  which  is  a 
minimizing  geodesic  from  a  to  6'  in  14  n  M;  and  its  projection  onto  N,  which  is 
a  straight  line.  These  conclusions  become  evident  under  the  following  important 
observation:  the  obstacle  W  defined  in  (2)  is  cylindrical  in  the  direction  of  N, 
i.e.,  X  £W  ii  and  only  if  x  -h  C  W. 

As  a  result  of  the  above  analysis,  we  can  focus  on  the  case  when  a  and  b  are 
aligned.  Without  loss  of  generality,  we  assume  that  a  and  b  both  belong  to  V. 
Since  any  minimizing  geodesic  between  such  a  and  b  is  contained  in  V,  we  can 
effectively  reduce  our  space  of  consideration  from  M  to  Mq  ^VdM,  which  is  a 
cross  section  of  M  and  two  dimensions  lower  than  M.  This  will  make  a  difference 
when  n  is  relatively  small. 

Remark  2.  Optimal  maneuvers  for  the  n-agent  system  can  be  alternatively 
viewed  as  the  outcomes  of  a  mechanical  experiment,  in  which  n  particles  of 
unit  mass  move  from  positions  ai, . . .  ,  an  on  a  plane  with  certain  initial  veloc¬ 
ities  and  no  external  force  acting  on  them.  In  this  interpretation,  the  result  in 
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Fig.  3.  Optimal  qJ  for  two  agents  case. 


Proposition  2  becomes  the  law  of  conservation  of  momentum.  See  [10]  for  further 
details. 

3.2  Two  Agents  Case 

Consider  the  simplest  case  when  n  =  2  with  aligned  a  =  (ai,  a2)  and  6  =  (6i,  62) 
such  that  ai  +  a2  =  +  ^^2  =  0-  If  o>*  =  (a*,  02)  ^  solution  to  problem  (3), 

then  Proposition  2  implies  that  and  02  (t)  are  symmetric  with  respect  to 
the  origin  for  all  t  G  [to,  t/j.  Hence  specifying  one  of  them,  say  oj,  is  sufficient  for 
describing  a*.  Moreover,  the  r-separation  constraint  can  be  formulated  as  the 
condition  that  aj  can  never  enter  the  open  ball  B{0,r/2)  of  radius  r/2  around 
the  origin.  By  noting  that  aj  and  give  identical  contributions  to  the  total 
energy,  we  finally  have  a  simplified  but  equivalent  version  of  problem  (3) : 

Find  the  energy  minimizing  curve  ai  in  \  B{0,  r/2)  joining  ai  to  61.  (6) 

Figure  3  shows  the  geometric  construction  of  a  solution  aj  to  problem  (6), 
which  is  a  geodesic  of  \  B(0,  r/2)  and,  depending  on  the  positions  of  ai  and 
bi,  may  contain  up  to  three  segments:  first  a  line  segment  from  ai  to  pi  tangent 
to  dB{0,r/2)  at  pi;  next  from  pi  to  qi  along  dB{0,r/2)]  and  finally  the  line 
segment  from  qi  to  61  tangent  to  dB(0,r/2)  at  qi.  The  case  when  61  is  “visible” 
from  fli  is  trivial, 

3.3  Three  Agents  Case 

The  case  n  =  3  is  more  complicated.  Figure  4  shows  all  the  possible  contact 
graphs  and  the  transitions  between  them,  with  the  “ground”  symbol  indicating 
that  there  is  a  transition  relation  with  state  1,  We  now  determine  the  geodesics 
in  each  one  of  the  discrete  states. 

Geodesics  in  state  1.  State  1  corresponds  to  the  contact  graph  of  three  isolated 
vertices,  hence  its  domain  Xi  corresponds  to  int{M),  the  interior  of  M.  By  the 
discussion  in  Sect.  3.1,  we  only  consider  Xi  =  V  nint(Af),  which  has  dimension 
4.  Geodesics  in  Xi  are  straight  lines. 

Geodesics  in  state  2,  3,  and  /.  States  2,  3  and  4  correspond  to  contact  graphs 
where  two  vertices  are  connected  to  each  other  and  the  third  one  is  isolated.  Let 
us  consider  state  2.  Its  domain  X2  is: 

^2  =  {(^1,2/1,  ^2, 2/2,  ^3, 2/3)  :  di2  =  r,  diz  >  r,  <^23  >r}nV 
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Fig.  4.  State  diagram. 


where  dij  =  denotes  the  distance  between  agent  i  and 

agent  j.  X2  has  dimension  3.  As  long  as  the  boundary  of  X2  is  not  reached, 
a  geodesic  in  X2  consists  of  a  constant  velocity  motion  for  agent  3  since  it  is 
“free” ,  while  the  motions  for  agents  1  and  2  are  determined  as  in  Sect.  3.2  for 
the  two  agents  case,  followed  by  a  possible  application  of  Proposition  1  if  their 
starting  and  destination  positions  are  not  aligned.  Similarly  for  X3  and  X^. 

Geodesics  in  state  5,  6,  and  7.  In  states  5,  6  and  7,  one  agent,  called  the  pivotal 
agent,  contacts  with  both  the  other  two  agents,  which  do  not  contact  each  other 
themselves.  The  domain  for  state  5  is: 

As  =  {(xi,yi,X2,y2,3:3,y3)  •  =  r,  dis  =  r,  ^23  >  r}  n  V: 

As  is  a  2-dimensional  submanifold  with  global  coordinates  (^125^13)  defined  by 

9i2  =  arctan  — — ^13  —  arctan  — — — . 

X2  -xi  xs-  Xi 

(^12,^13)  takes  values  in  [0,27r]  x  [0,27r]  with  opposite  edges  identified,  i.e.,  the 
2-torus  T^.  In  order  to  satisfy  the  constraint  ^23  >  the  shaded  region  (see 
Fig.  5)  has  to  be  removed  from  T^,  resulting  in  a  subset  As  homeomorphic 
to  X  (0,1).  So  topologically  As  (hence  As)  is  an  untwisted  ribbon  whose 
boundary  consists  of  two  disjoint  circles. 

Each  (^12,^13)  ^  As  determines  a  unique  point  /(^i2,^i3)  in  As  by 

f(0i2,9i3)  — -(-  COS012  —  cos  ^13,  —  sin ^12  -  sin  ^13,  2cos0i2  -  cos  ^13, 

3 

2  sin  9i2  —  sin  ^13,  —  cos  012  +  2cos0i3,  —  sin0i2  +  2sin0i3)^, 

which  is  an  embedding  of  As  into  R®.  The  standard  metric  on  R®  induces  by  / 
isometrically  a  metric  on  As-  A  curve  (0i2(t)5  ^i3(t))  Is  a  geodesic  in  As  under 
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Fig.  5.  The  domain  of  discrete  state  5. 


the  induced  metric  if  and  only  if  7(t)  =  f{9i2(t),eis(t))  is  a  geodesic  in  X^. 
Using  the  fact  that  7  is  a  geodesic  in  X5  if  and  only  if  its  acceleration  as  a  curve 
in  at  each  point  is  orthogonal  to  the  tangent  space  of  X5  ([5]),  we  obtain 
after  some  calculations  the  geodesic  equation  on  X5  as  (see  [11]  for  details) 

r  2^12  -  C0S(^12  -  ^13)^13  =  Sin(012  -  ^13)(^13)^ 

\  2^13  -  cos(0i2  -  ^13)^12  =  -  sin(0i2  -  ^13) (^12)^- 

There  are  certain  symmetries  in  equation  (7),  which  become  evident  by  writ- 
(^)  ill  new  coordinates  ^  =  ^12  +  ^13  and  77  =  O12  —  ^13,  leading  to: 

/  (2  -  COS  7])^  =  -$7)  sin  77 

l(2  +  cos77)f7=:  |((^)2  4-(?))2)sin77.  ^  ^ 

Integrating  the  first  equation  in  (8),  we  have 


^(2-cos77)  =  C2,  (9) 

for  some  constant  (72.  On  the  other  hand,  since  geodesics  have  constant  speed, 
there  exists  another  constant  Ci  such  that  ([11]) 

(2  -  cos77)(e)^  +  (2  +  cos  77)(77)2  =  4(7i. 

Substitution  of  (9)  into  the  above  equation  leads  to 


^^^2  _  8C1  -  Cl  -  4Ci  cos  77 
4  —  cos^  77  ’ 

which  together  with  (9)  governs  the  dynamics  of  77  and  ^  respectively. 
Geodesics  in  X^  and  Xj  can  be  obtained  similarly. 


(10) 


Geodesics  in  state  8  and  9.  Domains  X^^  Xq  and  Xj  share  a  common  boundary 
consisting  of  two  disjoint  circles,  which  form  the  domains  of  state  8  and  state  9 
respectively.  In  both  states,  the  three  agents  form  an  equilateral  triangle  centered 
at  the  origin.  The  only  difference  is  their  orientation.  Agents  1,  2  and  3  are 
numbered  counterclockwise  in  state  8  and  clockwise  in  state  9, 

Consider  state  8  and  its  domain  Xg.  Xg  is  a  one  dimensional  circle  and  can 
be  parameterized  by  (7,  which  is  the  angle  between  the  line  segment  joining  the 
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Fig.  6.  Concatenation  of  geodesic  segments. 


origin  to  agent  1  and  the  positive  ar-axis.  A  geodesic  in  Xs  in  this  coordinate 
must  then  be  of  the  form  (7(t)  =  tot  for  some  constant  angular  velocity  u. 

In  summary,  we  have  characterized  geodesic  segments  in  each  one  of  the 
discrete  states.  By  Theorem  1,  the  minimizing  geodesics  corresponding  to  the 
optimal  collision-free  maneuvers  for  the  three  agents  are  concatenation  of 
such  segments.  One  example  of  such  concatenations  is  shown  in  Fig.  6,  where 
the  starting  and  destination  positions  of  the  three  agents  are  marked  with  stars 
and  diamonds  respectively.  A  rod  exists  between  two  agents  if  and  only  their 
distance  at  the  corresponding  positions  is  r.  However,  it  should  be  pointed  out 
that  the  problem  of  finding  when  and  where  the  switches  between  geodesic  seg¬ 
ments  occur  remains  an  open  issue.  In  [11],  we  propose  a  numerical  procedure  to 
approximate  the  minimizing  geodesics  based  on  the  successive  optimization  of 
piecewise  linear  curves  in  Af .  At  each  iteration  a  convex  optimization  problem 
is  solved.  By  choosing  a  small  step  size  for  the  piecewise  linear  curves,  we  can 
obtain  a  reasonably  good  approximation. 

4  Conclusions 

The  problem  of  optimal  collision-free  maneuvers  for  multiple  agents  is  formulated 
and  shown  to  be  equivalent  to  the  problem  of  finding  minimizing  geodesics  in  a 
certain  manifold  with  boundary,  which  can  in  turn  be  interpreted  as  an  optimal 
control  problem  for  a  hybrid  system.  The  solution  is  given  for  the  two  agents  case. 
For  the  three  agents  system  we  derive  the  dynamics  of  the  segment  of  optimal 
maneuver  associated  to  each  discrete  state.  The  overall  optimal  maneuver  is 
shown  to  be  a  concatenation  of  such  segments. 
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Abstract.  A  multiple  model  based  observer/estimator  for  the  estima¬ 
tion  of  parameters  is  used  to  reset  the  parameter  estimation  in  a  con¬ 
ventional  Lyapunov  based  nonlinear  adaptive  controller.  The  advantage 
of  combining  both  approaches  is  that  the  performance  of  the  controller 
with  respect  to  disturbances  can  be  considerably  improved  while  a  re¬ 
duced  controller  gain  will  increase  the  robustness  of  the  approach  with 
respect  to  noise  and  unmodeled  dynamics.  Several  alternative  resetting 
criteria  are  developed  based  on  a  control  Lyapunov  function. 


1  Introduction 

The  use  of  multiple  models  to  switch  or  reset  parameter  estimators  has  been 
proposed  in  order  to  speed  up  the  convergence  rate  of  certainty  equivalence 
adaptive  control  of  linear  systems  [1,2, 3,4, 5, 6, 7, 8]. 

In  this  paper  we  present  a  hybrid  approach  to  speed  up  transients  in  con¬ 
tinuous  Lyapunov  based  nonlinear  adaptive  control  systems.  Hereby,  a  multiple 
model  observer  (MMO)  is  used  to  reset  the  parameter  estimation  in  a  nonlin¬ 
ear  adaptive  controller.  The  advantage  of  combining  both  approaches  is  that 
transients  due  to  adaptation  can  be  damped  out  while  the  performance  of  the 
controller  with  respect  to  disturbances  can  be  improved.  As  a  consequence  the 
gain  of  the  continuous  adaptive  controller  can  be  considerably  lowered  thus,  in¬ 
creasing  the  robustness  of  the  approach  with  respect  to  noise  and  unmodeled 
dynamics.  The  parameter  resetting  is  based  on  a  Control  Lyapunov  function 
and  can  guarantee  asymptotic  stability.  The  main  contributions  of  the  paper  are 

—  an  extension  of  multiple  model  based  adaptive  control  to  the  class  of  para¬ 
metric  strict  feedback  nonlinear  systems, 

—  the  formulation  of  a  set  of  sufficient  closed  loop  stability  conditions  for  re¬ 
setting  tuning  function  based  nonlinear  adaptive  controllers, 

—  the  introduction  of  a  fast  multiple  model  observer,  from  which  even  under 
transient  conditions  an  accurate  parameter  estimate  can  be  obtained. 

M.D.  Di  Benedetto,  A.  Sangiovanni-Vincentelli  (Eds.):  HSCC  2001,  LNCS  2034,  pp.  319-332,  2001. 
(c)  Springer- Verlag  Berlin  Heidelberg  2001 
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The  paper  is  organised  as  follows:  In  Section  2  some  results  of  constructive 
nonlinear  adaptive  control  are  briefly  reviewed  and  a  motivation  for  discontin¬ 
uous  parameter  resetting  is  given.  This  is  followed  by  an  analysis  of  the  closed 
loop  stability  implications  of  resetting  parameter  estimates  (Section  3)  where  a 
first  order  and  a  second  order  example  are  used  to  illustrate  the  results.  Sec¬ 
tion  4  describes  the  concept  of  multiple  model  observers  and  gives  for  a  special 
plant  structure  sufficient  conditions  for  stability  of  parameter  resetting.  At  the 
end  discussions  of  a  first  order  system  as  an  application  of  the  method  and  some 
simulation  results  are  given. 

2  Nonlinear  Adaptive  Backstepping 

Consider  the  adaptive  tracking  problem  for  a  parametric  strict-feedback  sys¬ 
tem  [9] 

Xi  =  X2  (1) 


Xn-1  =Xr^+(pn-l{Xl,X2,...  ,Xn-l)'^0 
Xn  =  13{X)U  -h  ipn{x)'^9 
y  =  xi 

where  ^  G  is  a  vector  of  unknown  constant  parameters,  /?  and  F  = 
[</?!,...  ,(pn]  are  smooth  nonlinear  functions  taking  arguments  in  It  has  been 
shown  that  in  a  tuning  function  adaptive  controller  for  such  a  system  the  adap¬ 
tive  control  law  and  the  parameter  update  law  take  the  following  form 

1 


u  = 


P(x) 

where  yj.  is  the  reference  signal  to  be  tracked  by  the  output  y 


Vr^  =  (yr,3/r,--  - 


The  control  law  and  the  tuning  functions  are  given  recursively  by 


Zi=Xi- 


Oii-l 


i  — 1 


k=l 


dy 


(fc-i) 


i— 1 


dO 


k=2 


dO 


ri{xi,e,y^^  Ti^i  +  WiZi 

/_  A 

w 


da. 


i  =  1 ..  .n 


(2) 

(3) 

(4) 

(5) 


(6) 

(7) 

(8) 
(9) 
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where  xi  =  (xi, . . .  ^Xi),  ao  =  0,  tq  =  0,  ci  >  0.  The  control  law  together  with 
the  parameter  update  law  render  the  time  derivative  of  the  Lyapunov  function 

V„  =  \z'^z  +  \¥r^H  with  9  =  9-9  (10) 

Zt  z 

negative  semidefinite  along  trajectories  of  the  closed  loop  system: 

n  n 

Vn  = -'^Ckzl-'Y^Ki\wi\^zf  <-co\z\^  where  co  =  ^min^Ci  (11) 

A:=l  k=i 

Our  main  objective  is  to  improve  the  transient  performance  of  the  closed  loop 
system,  in  particular  with  respect  to  the  unknown  parameter  vector  0  which  is 
assumed  to  be  constant  with  respect  to  time. 

It  is  a  well  known  fact  that  for  this  adaptive  control  schemes  the  transient 
performance  can  be  improved  by  increasing  any  of  the  design  parameters  Ci, 
Ki  and  r.  The  higher  the  gain  the  faster  the  transient  response  of  the  control 
systems.  In  practical  applications  however,  high  gain  should  be  avoided  as  there 
are  always  unmodelled  dynamics  or  even  time  delays  (related  to  computer  im¬ 
plementation)  in  the  system  which  may  lead  to  instability  if  the  loop  gain  is  too 
high.  Thus,  other  strategies  of  counteracting  uncertainties  are  highly  desirable. 

Such  a  strategy  is  provided  by  the  multiple  model  switching  and  tuning 
approach,  where  the  estimates  are  taken  from  a  finite  set 

9i,  2  =  1,. . .  ,iV. 

The  multiple  model  observer  provides  additional  information  on  parameter  un- 
certainies  which  can  then  be  used  to  instanteneously  reset  the  parameter  esti¬ 
mate  9,  Suppose  the  best  estimate  of  the  multiple  model  observer  with  respect 
to  prediction  performance  is 


0+  =  9j. 

Then  a  decision  has  to  be  made  whether  or  not  to  use  this  additional  information. 
In  the  case  when  the  multiple  model  estimate  is  used  the  current  continuous 
estimate  9~  will  be  discarded  and  the  continuous  update  law  reset  to  the  new 
value.  This  resetting  decision  should  not  be  based  on  the  modelling  performance 
alone.  It  should  also  be  guaranteed  that  the  control  performance  and  in  particular 
the  transient  behaviour  is  improved  via  resetting. 

In  between  the  resetting  events  the  parameter  estimate  will  still  be  governed 
by  the  adaptation  law  and  it  will  thus  be  piecewise  continuous.  This  will  result 
in  discontinuous  control  and  adaptation  laws.  Since  the  state  transformation  in 
Eq.  (5)  is  parameterised  by  9  the  states  22, . . .  ,  will  be  discontinuous  in  time. 

In  the  remainder  of  the  paper  the  implications  of  such  a  resetting  strategy 
will  be  studied. 
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3  Stability  Analysis  of  Parameter  Resetting 

3.1  Sufficient  Conditions  for  Stability 

Stability  results  for  discontinuous  Lyapunov  functions  exist,  e.g.  [10].  For  stabil¬ 
ity  it  is  sufficient  that 

1.  V{x)  be  continuous  with  respect  to  its  arguments 

2.  V{x)  is  non-increasing  along  trajectories  in  between  switching  events, 

3.  F(x+)  <  V[x~)  whenever  there  is  a  jump  from  x~  =  lim^tn.  x{t)  to  x'^  = 
limt|t*  x{t)  at  some  time  instant  t*. 

Consider  the  Lyapunov  function  (10)  of  the  tuning  function  approach 

+  with  e  =  e-9.  (12) 

For  the  tuning  function  approach  it  can  be  easily  shown  that  properties  1  and  2 
hold  due  to  the  stability  of  the  closed  loop  system  when  no  resetting  is  applied. 
When  the  parameter  estimate  0  is  reset,  the  state  variable  2:  depending  on  0 
changes  discontinuously  with  time.  Then,  to  obtain  a  sufficient  condition  for 
stability  it  remains  to  be  analysed  whether 

=  V4z{e+),ej+)  -  <  o  (13) 

holds.  If  this  is  the  case  then  a  resetting  of  0  from  0~  to  9'^  is  admissible.  In 
general  the  state  vector  2:  will  depend  on  ^  in  a  nonlinear  way.  In  order  to  develop 
some  stability  criteria  the  following  assumption  may  be  made  (it  will  be  shown 
in  later  sections  how  this  can  be  replaced  by  other  assumptions): 

Assumption  3.1.  Set  the  step  change  in  parameter 

Ae^9^-9~.  (14) 


There  exist  a  matrix- valued  function  M{z~  ,9~  ^^)  such  that 

[z+f  {z+)  <  {z-  +  MAey  (2-  +  MAe^  (15) 

for  all  Ae^DC  Rp. 

Under  assumption  3.1  the  following  bound  on  the  step  change  of  the  Lyapunov 
function  (10)  can  be  given: 


/iy.  =  (2+)  +  («>  -  e+Y  r-'-  (e  -  0+ j 


ziy„  <2 


m'^z-  -  r-^e-  A§  +  Ae'^  [m'^m  +  r“‘]  ao 


(16) 

(17) 
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9-  ^e~e- 


For  positive  definite  +r~^  >  0  the  sufficient  condition  for  stability  AVn  < 

0  is  satisfied  inside  the  hyper-ellipse 


r-^9- 


A9  +  AB^  [M'^M  +  Ae  =  ^ 


(18) 


It  can  be  easily  verified  that  even  in  the  case  when  0  steps  from  6  to  the 
correct  parameter  value  9'^  =  9  the  condition  for  stability  is  not  necessarily 
satisfied  because  in  this  case  the  requirement  would  be: 

2{z-)'^M9-  +  {9-f{M^M  -  r-^)9-  <  0.  (19) 


It  has  been  shown  above  that  the  set  of  admissible  parameter  changes  A9 
depends  on  the  state  2  and  on  the  parameter  error  9.  While  2”  and  z"*"  can  be 
computed,  additional  information  on  the  estimation  error  is  necessary  to  check 
the  admissibility  of  A9.  In  the  remainder  of  the  paper  two  ways  of  obtaining  the 
required  knowledge  of  9  will  be  presented.  The  first  approach  is  by  exploiting 
properties  of  the  closed  loop  system  while  the  second  approach  uses  additional 
information  supplied  by  an  multiple  observer. 


3.2  Reference  Trajectory  Resetting 


The  condition  (13)  on  AV  can  be  considerably  simplified  when  resetting  of  the 
reference  trajectory  Pr  is  used  in  combination  with  the  parameter  resetting. 

Reference  trajectory  resetting  can  be  applied  most  easily  in  the  case  where  pr 
and  its  derivative  are  generated  by  a  linear  reference  model  which  is  driven  by 
some  external  reference  input  signal  r{t).  For  the  following  calculations  we  as¬ 
sume  the  existence  of  a  reference  model  since  the  states  of  such  a  system  can 
be  reset  directly.  In  the  other  case  where  Pr  and  its  derivatives  are  generated 
externally  the  reset  is  accomplished  by  modification  of  the  reference  signal  us¬ 
ing  the  output  <5, of  an  additional  linear  asymptotically  stable 
autonomous  system  ■ 

Reference  trajectory  initialisation  is  originally  a  tool  for  improving  the  tran¬ 
sients  in  adaptive  tuning  function  control  systems  [9].  In  fact,  by  resetting  the  n 
values  Pr{t^),  Pr{t'^),  •  •  •  ,  an  additional  degree  of  freedom  is  obtained 

which  enables  us  to  set  z+  =  0.  From  Eq.  (5)  it  can  be  seen  that  z+  =  0  requires 
the  solution  of  set  of  equations 


=  Xi  -  ,  Xi_i,9+,yr{t+), . . .  i  ==  1, . . .  ,n 

(20) 


It  can  be  shown  [9]  that  the  solution  to  these  equations  does  not  depend  on  the 
controller  parameters. 
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The  step  change  in  the  Lyapunov  function  with  reference  trajectory  resetting 
is 

AV„  =  {$-  9+Y  r-^  (e  -  9+)  -  {z-f  (z-)  -{9-  tf  (9  -  9-) 

=  AFr-^A9-2(9-Y  r-^A9- {z-f  {z-)  (21) 

for  which  we  can  obtain  a  controller  independent  upper  bound 

AVn<A9‘^r-^A9-2(9Y  r-'^A9  (22) 

When  trajectory  resetting  is  used,  the  Lipschitz  assumption  3.1  (where  M  might 
be  difficult  to  compute)  is  no  longer  required  because  2:"^  =  0  in  Eq.  (16). 

3.3  Application  to  a  First  Order  System 

Consider  the  tracking  control  of  the  first  order  system 

ii=^\{x\)e  +  u  (23) 

An  adaptive  tuning  function  controller  is  simply 

u  =  -Lpi{xi)e  -  ClZi  -  ijr  (24) 

e  =  ^ziipi{xi)  =7x1  (25) 

Zi^Xx-  yr 

This  controller  based  on  the  control  Lyapunov  function 

renders  the  derivative  of  the  Lyapunov  function  negative  semi-definite 

V  =  ~cizf  <  0. 

The  closed  loop  system  is  given  by 

=  -Cizi  -h(pi(xi)0  (27) 

The  time  derivative  of  the  squared  error  along  the  solution  of  (27)  is 

zf^  =  zizi  =  -cizf  +  zi(pi(xi)0  (28) 

For  the  rest  of  the  discussion  of  the  first  order  case  we  assume  that  (pi(xi)  >  0. 
This  assumption  is  not  necessary  for  the  approach  in  general  but  it  simplifies 
the  switching  law  considerably. 


_d 

dt 


G 
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For  the  first  order  system  (23)  and  the  Lyapunov  function  (26)  we  obtain  by 
use  of  Eq.  (16)  the  following  sufficient  stability  condition: 

AV  =  V^  -  V-  ^  ^  -  ff-'j  (0+  -  (?-))  <  0  (29) 

This  gives  the  following  bounds  on  the  step  change  in  the  parameter  estimate: 

sgn  =  sgn  (30) 

|^^|<2|0-|  (31) 

In  general,  condition  (31)  cannot  be  verified  without  additional  information 
on  the  parameter  estimate.  However  a  switching  law  S(zi,A0)  can  be  designed 
such  that  condition  (30)  holds. 

Using  this  switching  law  the  parameter  resetting  law  is  constructed  in  the 
following  way 

S  =  0- +  ^0+ -  ff-j  S(zi,A0)  =0-  +  A0  S(zi,A0)  (32) 

where  5  assumes  the  values  1  or  0  according  to  the  following  set  of  inequalities 

!zi  >  ei  A  A0  >  £2 
V 

zi  <  ~€i  A  AO  <  —£2 

5  —  0  elsewhere  (33) 

Condition  (32)  states  that  resetting  occurs  whenever  the  magnitude  of  the 
control  error  zi  exceeds  some  threshold  and  at  the  same  time  there  is  a  signif¬ 
icant  discrepancy  between  continuous  parameter  estimate  and  multiple  model 
parameter  estimate  having  the  same  sign  as  the  control  error. 

Note  that  due  to  the  assumption  that  (p  is  always  positive  we  obtain  from 
the  closed  loop  error  equation  (27): 

zizi  >0  implies  sgn(ii)  =  sgn(^)  (34) 

Thus,  provided  that  is  increasing  while  it  crosses  the  threshold  £i  the  sign 
of  ii  is  a  direct  indicator  of  the  sign  of  the  parameter  error  9.  In  the  general  case, 
the  sign  of  (f  will  be  known  and  the  resetting  law  can  be  modified  accordingly. 
This  leads  us  to  the  following  theorem 

Theorem  3.2.  1.  Consider  the  first  order  system  (23)  together  with  the  con¬ 
tinuous  control  law  (24)  and  the  update  law  (25).  Assuming  >  0, 

7  >  0  and  Cl  >  0.  If  the  parameter  9  is  reset  under  the  condition 

2isgn(ii)  =  ^1  /\  ziA9  >  £i£2,  £1  >  0,  £2  >  0  (35) 

then,  the  sign  condition  (30)  is  satisfied. 
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2.  Provided  the  sign  condition  is  satisfied,  then  a  decrease  ofV  in  Equation  (26) 
at  the  smitching  instant  is  obtained  provided  that 

A9  <  2  9-  (36) 

holds.  Thus  a  sufficient  condition  for  stability  is  satisfied. 

3.  If  to  the  contrary 

A9  >  2  9-  (37) 

holds  then  the  control  error  zi  is  driven  towards  zero  as  long  as  |2:i|  >  £:i 
despite  of  the  increase  in  value  ofV. 


Proof  The  first  and  second  part  of  the  Theorem  has  been  proven  above. 

If  the  assumptions  of  the  third  part  of  the  theorem  hold  then,  outside  \zi\  >  6i 
we  have  along  the  solutions  of  the  closed  loop  equation: 


_d 

dt 


=  zizi 


=  -Cizf  +  Zi^i(xi)  9  -  Aes{y,  A6)^ 

<  -cizi  +  |ii¥>i(xi)|  6-  -  <  0  (38) 


due  to  (37)  which  implies  that  zi  is  driven  towards  the  origin.  ■ 

As  a  remark,  one  might  note,  that  case  3  of  Theorem  3.2  implies  stability  but 
possibly  with  reduced  transient  performance  and  chattering. 

The  negative  jump  in  the  Lyapunov  function  could  be  interpreted  as  im¬ 
proved  transient  performance.  This  follows  from  the  dependency  of  transient 
performance  of  the  tuning  function  approach  on  the  initial  conditions  which  has 
been  analysed  in  [9]. 


3.4  Application  to  a  Second  Order  System 

Consider  the  second  order  system  with  one  parameter 

±1  =  X2-h^(xi)9 

X2  =  u,  (39) 

Designing  the  tuning  function  controller  (2)  for  such  a  system  requires  one 
backstop.  Assuming  that  the  parameter  estimate  9  can  vary  discontinuously  with 
time  we  will  thus  have  also  discontinuous  changes  with  time  in  ai  and  Z2  and  in 
the  corresponding  Lyapunov  function  V  =  ^z^ ^9“^ .  The  step  change 

in  the  Lyapunov  function  can  be  expressed  as 

=  V+  -  1/-  =  z^y^i{xi)A0  +  l-cpl{xi)A§^  -  U-A0  +  ^AB^ 

2  7  27 

=  ^  (^  +  AB'^  -  ^^0-  -  Z2  AB  (40) 
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This  corresponds  with  Assumption  3.1  and  Eq,  (15)  where 

=  z"  +  MAe 

The  reset  conditions  discussed  in  sections  3.3  and  3.4  require  the  information 
whether  the  states  of  zi  and  Z2  cross  some  threshold  from  above  or  below.  No 
explicit  knowledge  of  the  derivatives  of  the  states  is  required.  In  case  of  noisy 
state  measurement  multiple  crossing  of  the  threshold  may  occur,  however,  by 
imposing  an  additional  threshold  on  A9  a  hysteresis  is  introduced  and  chattering 
cannot  occur. 

4  Multiple  Model  Observer  (MMO) 

As  explained  above  a  multiple  observer  approach  can  be  used  to  avoid  large 
transient  errors  in  continuous  adaptive  control.  Quite  similar  to  the  multiple 
model  estimation  described  in  [2, 3, 4, 7],  the  idea  is  to  construct  a  finite  set  of 
parallel  observers  each  of  which  is  designed  for  a  fixed  parameter  value.  In  its 
simplest  form  the  MMO  constists  of  a  set  O  oi  N  individual  observers  o*  each 
parameterised  with  a  fixed  parameter  value  9i.  All  N  observer  cover  the  range  of 
admissible  parameter  values.  Figure  (1)  shows  the  structure  of  a  multi-observer 
parameter  estimation.  Each  of  the  N  observer  estimates  the  states  of  the  system 
and  is  driven  by  the  residual  eu  =  Xi  ~  xu.  Since  any  mismatch  between  a 
single  observer  and  the  physical  system  will  in  general  lead  to  a  steady-state 
estimation  error,  this  error  can  be  used  to  determine  the  best  observer  for  the 
actual  system. 

Using  discontinuous  output  injection  functions  is  common  in  sliding  mode 
observers  [11].  A  hybrid  observer  using  convergence  information  to  switch  be¬ 
tween  several  discontinuous  output  injection  functions  for  nonlinear  systems  has 
been  reported  in  [12].  Here,  we  propose  instead  to  use  a  set  of  observers  with 
fixed  output  injection  functions  which  can  have  considerably  faster  transients. 

A  performance  index  Qi(xi,y)  is  defined  for  each  observer  of  the  set  O.  The 
performance  index  weighs  the  output  error  of  the  observer,  thus  quantifies  the 
mismatch  between  the  plant  and  the  individual  observer.  A  switching  logic  L 
is  used  to  determine  the  estimate  9i  of  the  multi-observer  O.  L  satisfies  two 
purposes: 

1.  selecting  the  coefficient  9i  corresponding  to  the  observer  Oi  with  the  best 
performance. 

2.  providing  a  mechanism  that  ensures  a  convergence  of  the  estimator  after  a 
finite  number  of  switches. 

In  order  to  prevent  chattering,  two  different  approaches  have  been  suggested 
in  literature 


328  J.  Kalkkuhl  et  al. 


Fig.  1.  Multiple  model  observer  parameter  estimation 


—  Dwell  time  switching  [5]  where  after  each  switch  for  a  certain  period  of  time 
switching  is  prohibited. 

-  Hysteresis  switching  [1,13]:  Let  Op  be  the  valid  observer  at  time  then  a 

switch  to  a  new  observer  oi  occurs  only  if  +  h)  <  where 

Qp{t^)  is  the  current  performance  of  the  observer  Op  and  >  0  is  the 

hysteresis.  Otherwise  no  switching  will  occur  and  Op  will  remain  valid. 


4.1  Construction  of  the  Individual  Observers  in  the  First  Order 
Case 

Consider  the  system  (23)  where  the  parameter  9  is  treated  as  an  augmented 
state 


^1  =  <fi{y)0  +  u 

(9  =  0 

y  =  xi  (41) 

It  is  assumed  that  (pi{y)  >  0  and  that  the  parameter  9  is  contained  in  a  closed 
interval  The  interval  is  discretised  using  a  set  of  N  parameter  val¬ 
ues  9 min  <  ^1  <  ^2  <  •  •  ■  <  <  Omax-  Each  of  the  N  individual  observers  of  the 

multiple  model  observer  will  be  centered  around  one  of  the  discrete  parameter 
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values  9i.  For  this  purpose  Eq.  (41)  is  rewritten  into 


+  u 

X2i  =  0  (42) 


where  X2i  =  9  —  9i.  Following  the  Lyapunov  based  observer  design  in  [14]  we 
propose  to  use  the  following  individual  nonlinear  observer 

xii  =  ^\{y)9i  +  -  xu)  +  u  +  (pi(y)x2i 

X2i  =^(^‘^(pi{y){y  -  Xu),  u>0,  (43) 

Defining  the  error  ei  =  [eu,  e2i]^  =  \y-  -  X2if  the  observer  will  result 

in  the  bilinear  error  dynamics 

e<  =  ¥’(i/)  (_^‘^j)e,.  (44) 

A 


where  the  matrix  A  is  Hurwitz  and  ip(y)  represents  the  nonlinearity  in  the  system 
output.  The  observer  design  renders  the  derivative  of  the  Lyapunov  function 


V,{ei) 


ei 


(45) 


negative  definite  Vi  =  —2u}(p{y)el^  <  0. 

An  important  property  of  the  error  differential  equation  (44)  is  that  its  so¬ 
lution  can  be  explicitly  given.  Knowing  the  measurable  output  error  eu(t  ~  T) 
and  eu(t)  at  some  time  instant  t  the  parameter  estimation  error 

e2i(t)  =  ^  [(1  +  eH(*)  -  e-‘"«'eH(i  -  T)]  (46) 

can  be  determined,  where  y*{t  —  T,t)  =  ^i{y{x))dT  >  0.  Thus,  even  under 
observer  transients  a  parameter  estimate 

9i  =  9i-{-  X2i{t)  +  e2i{t)  (47) 


can  be  computed. 

Anti-windup  is  introduced  for  the  observer  state  X2i  by  defining  the  local 
bounds  9i.  The  state  equation  X2i  is  set  to  zero  if  X2i  -h  9i  ^  [9i-i,9i]  and 
{y  -  xu)x2i  >  0.  Hence,  only  one  individual  observer  will  have  an  output  error 
converging  to  zero  and  consequently  a  cost  index  Qi  converging  to  zero  indepen¬ 
dently  of  the  particular  cost  index  that  is  used. 

The  properties  of  the  MMO  can  be  used  to  derive  the  following  resetting  law: 


Theorem  4.1.  Consider  the  control  system  (23)  together  with  the  control 
law  (24),  the  parameter  update  law  (25)  and  the  MMO  (4^).  Suppose  that  Oi 
is  the  observer  that  has  been  selected  according  to  the  cost  index.  Then,  set¬ 
ting  9'^  —  9i  will  result  in  a  negative  step  of  the  Lyapunov  function  (26)  if 
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1-  does  not  saturate  within  the  time  intervall  r  G  [t  —  T,t]. 

2.  (9i_i  <0i<Ji.  _  _ 

3.  either  (a)  §-  -9i>ei-  9i  or  (b)  6»i_i  -  9-  >  9i  -  0i_i. 

Proof.  If  condition  1  of  the  theorem  holds,  according  to  Eqs.  (46)  and  (47)  we 
have 

9i  =  9i  +  X2i{t)  +  e2i{y*(t,t-  T),  eu{t),  eu{t  -  T)).  (48) 

If  in  addition  to  this,  condition  2  is  satisfied,  then  it  can  be  implied  that  the  real 
parameter  is  contained  in 

<e<Si.  (49) 

From  condition  3  it  follows  that  either  3a  is  satisfied  in  which  case  we  obtain  by 
adding  6  to  both  sides,  rearranging  and  employing  (49) 

-A9  =  e~  ^ei<  2(0  ~  Si)  <  2{e-  ~e)  =  -20-  (50) 

If  on  the  other  hand  3b  is  satisfied  then  by  subtracting  0  from  both  sides  and 
employing  (49) 

A0  =  0i~0-  <  2{0i_i  -  0~)  <  2{0  -  0-)  =  20- ,  (51) 

Consequently,  conditions  (30)  and  (31)  are  satisfied  which  is  sufficient  for  sta¬ 
bility.  ■ 

Note  that  the  MMO  approach  does  not  rely  on  assumption  3.1. 

5  First  Order  System 

Consider  the  first  order  system  (41)  where  ipi{xi)  =  x\  together  with  the  control 
law  (24)  and  the  update  law  (25).  The  design  of  the  MMO  (43)  is  done  by  using 
five  parameter  hypotheses  0i  G  {-10,  -5,0,5, 10}.  The  parameter  estimate  0  is 
reset  if  the  Theorem  4.1  together  with  (32)  hold.  The  simulation  results  with 
and  without  parameter  resetting  are  depicted  in  Figure  (2).  Consider  the  sim¬ 
ulation  scenario  where  the  system  should  follow  a  ramp  signal  with  the  slope 
0.1sec“^  The  parameter  0  jumps  at  time  t  =  4sec  from  ^  =  9  to  0  =  -8  and 
at  time  t  =  7sec  to  0  =  4.  White  noise  is  distributed  to  the  system’s  output. 
Note  that  the  scenario  differs  slightly  from  the  above  theoretical  considerations 
where  the  parameter  0  is  assumed  to  be  time  invariant.  The  upper  left  picture 
in  Figure  (2)  shows  the  control  error  for  both  cases  with  (fat  black  line)  and 
without  (gray  line)  using  the  MMO.  The  upper  right  picture  shows  the  control 
signal  respectively.  The  lower  left  picture  depicts  the  real  parameter  value  0  (dot¬ 
ted),  the  estimate  of  the  MMO  0i  (dashed  gray),  the  estimate  0  with  parameter 
resetting  (solid  fat)and  0  without  resetting  (dashed  fat  line).  Using  the  MMO 
estimation,  S  converges  faster  to  the  real  parameter  value  and  the  control  error  is 
removed  faster.  The  lower  right  picture  of  Figure  (2)  shows  the  faster  decrease  of 
the  Lyapunov  function  (26)  and  the  performance  enhancement.  The  simulation 
shows  an  improved  performance  even  for  step  disturbances  in  the  parameter. 
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Fig.  2.  First  order  example 


6  Conclusions 

The  presented  paper  provided  an  extension  of  multiple  model  based  adaptive 
control  to  the  class  of  parametric  strict  feedback  nonlinear  systems.  As  a  main 
contribution  a  set  of  sufficient  closed  loop  stability  conditions  for  resetting  tuning 
function  based  nonlinear  adaptive  controllers  was  given.  Also,  a  fast  multiple 
model  observer  was  introduced,  from  which  even  under  transient  conditions  a 
parameter  estimate  can  be  obtained.  A  first  order  control  example  showed  that 
recovering  of  the  control  error  can  be  improved  after  instantaneous  changes  of 
the  parameter. 

Future  work  will  be  dedicated  to  the  application  of  multiple  observers  in  au¬ 
tomotive  wheel  slip  control  where  a  fast  recovery  of  wheel  slip  after  instantaneous 
changes  of  the  tyre/road  friction  coefficient  is  required. 
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Abstract.  In  many  control  applications,  a  specific  set  of  output  tracking 
controllers  of  satisfactory  performance  have  already  been  designed  and 
must  be  used.  When  such  a  collection  of  control  modes  is  available,  an 
important  problem  is  to  be  able  to  accomplish  a  variety  of  high  level  tasks 
by  appropriately  switching  between  the  low-level  control  modes.  In  this 
paper,  we  define  a  concept  of  control  modes,  and  propose  a  framework  for 
determining  the  sequence  of  control  modes  that  will  satisfy  reachability 
tasks.  Our  framework  exploits  the  structure  of  output  tracking  controllers 
in  order  to  extract  a  finite  graph  where  the  mode  switching  problem 
can  be  efficiently  solved,  and  then  implement  it  using  the  continuous 
controllers.  Our  approach  is  illustrated  on  a  helicopter  example,  where 
we  determine  the  mode  switching  logic  that  achieves  the  high-altitude 
takeoff  task  from  a  hover  mode. 


1  Introduction 

Large  scale  systems  like  automated  highway  systems,  air  traffic  management 
systems,  unmanned  aerial  vehicles  are  multi- agent,  multi-objective  systems  that 
operate  in  many  modes  of  operation.  This  results  in  systems  of  very  high  com¬ 
plexity  which  may  dramatically  limit  the  applicability  of  current  analysis  and 
design  methods.  A  natural  way  to  reduce  the  complexity  of  system  design  uses 
compositional  methods  which  solve  a  complex  problem  by  decomposing  it  into 
a  sequence  of  smaller  problems  of  manageable  complexity.  For  example,  in  so¬ 
phisticated  flight  management  systems  [3],  modern  aircraft  fly  from  origin  to 
destination  while  satisfying  a  large  number  of  aerodynamic,  scheduling,  and  air 
traffic  constraints  by  switching  among  a  finite  set  of  flight  modes,  where  each 
flight  mode  essentially  corresponds  to  a  different  output  tracking  controller. 

More  generally,  given  a  continuous  control  system,  a  control  mode  is  defined 
as  the  operation  of  the  system  under  a  controller  that  is  guaranteed  to  track  a 
certain  class  of  output  trajectories.  Different  outputs  of  interest  correspond  to 
different  control  modes.  Given  a  set  of  control  modes,  the  mode  switching  prob¬ 
lem  attempts  to  find  a  finite  sequence  of  the  control  modes  as  well  as  switching 
conditions  in  order  to  satisfy  various  tasks.  In  this  paper,  we  focus  on  reachability 
tasks. 

M.D.  Di  Benedetto,  A.  Sangiovanni-Vincentelli  (Eds.);  HSCC  2001,  LNCS  2034,  pp.  333-346,  2001. 
(c)  Springer- Verlag  Berlin  Heidelberg  2001 
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Problem  1.  Given  a  control  system  and  a  finite  set  of  control  modes  for  the 
system,  determine  whether  there  exists  a  finite  sequence  of  modes  that  will  steer 
the  system  from  an  initial  control  mode  to  a  desired  final  control  mode.  If  such 
a  sequence  exists,  then  determine  the  switching  conditions. 

Clearly,  in  this  setup,  many  more  interesting  problems  can  be  formulated.  For 
example  one  can  ask  what  are  the  optimal  switching  conditions,  where  optimality 
can  mean  minimum  time,  or  minimum  number  of  switchings.  Furthermore,  one 
can  ask  whether  a  set  of  modes  is  sufficient  for  performing  a  reachability,  or 
more  general,  task.  In  this  paper,  we  focus  on  Problem  1,  while  setting  up  the 
framework  for  considering  these  more  general  questions  in  the  future. 

In  its  full  generality,  Problem  1  can  be  tackled  using  controller  synthesis 
methods  for  hybrid  systems  [1,7,12,14].  However,  termination  conditions  for  such 
synthesis  procedures  are  limited  [6],  and  the  computational  complexity  of  such 
procedures  could  be  prohibitive  due  to  nested  reachability  computations.  It  is 
therefore  evident  that  in  order  to  scale  our  methods  to  real-life  examples,  struc¬ 
ture  must  be  imposed  on  the  system,  and  subsequently  exploited  in  our  analysis 
and  synthesis  methods. 

In  order  to  reduce  the  complexity  of  the  mode  switching  problem,  we  start 
by  assuming  that  output  tracking  control  laws  have  been  designed  for  each  con¬ 
trol  mode.  Feedback  greatly  simplifies  the  continuous  models  in  each  discrete 
location  since  the  complexity  of  the  continuous  behavior  is  now  reduced  to  the 
complexity  of  the  trajectories  we  design.  Therefore,  many  reachability  compu¬ 
tations  that  are  required  in  our  approach  can  be  greatly  simplified  by  properly 
designing  the  desired  trajectories.  Even  though  feedback  control  simplifies  the 
continuous  complexity,  the  problem  of  having  nested  reachability  computations 
is  still  present.  In  order  to  avoid  such  expensive  computations,  we  place  a  con¬ 
sistency  condition  in  our  mode  switching  logic  which  is  reminiscent  of  the  notion 
of  bisimulation.  We  propose  an  algorithm  which  given  an  initial  set  of  control 
modes,  constructs  a  control  mode  graph  which  refines  the  initial  control  modes 
but  is  consistent.  Construction  of  the  mode  graph  can  be  done  off-line  or  every 
time  a  new  control  mode  is  designed,  allowing  the  mode  switching  problem  to 
be  efficiently  solved  on-line,  in  real  time. 

2  Problem  Formulation 

Throughout  this  paper,  we  consider  a  nonlinear  system  modeled  by  differential 
equations  of  the  form 

x{t)  =  f{x{t))  +  g{x{t))uit),  x{to)  =  xo,  t>  to  (2.1) 

where  a:  G  w  6  R^,  f{x)  :  R^  ^  R^  and  g{x)  :  R’^  ^  R^  x  R^.  The  system  is 
assumed  to  be  as  smooth  as  needed.  We  now  define  a  concept  of  control  mode. 

Definition  1  (Control  Modes).  A  control  mode,  labeled  by  qi  where  i  G 
is  the  operation  of  the  nonlinear  system  (2.1)  under  a  closed-loop 
feedback  controller  of  the  form 
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u{t)  =  ki{x{t),ri{t))  (2.2) 

associated  with  an  output  yi{t)  =  hi{x{t))  such  that  yi{t)  shall  track  ri(t)  where 
yi{t),  ri{t)  e  hi'.W^  x  R”"^  ->  W  for  each  ie  N}. 

We  assume  that  ri  £  Hi,  the  class  of  output  trajectories  associated  with  the 
control  mode  qi,  when  the  initial  condition  of  the  system  (2.1)  starts  in  the  set 
Si{ri)  C  Xi,  output  tracking  is  guaranteed  and  the  state  satisfies  a  set  of  state 
constraints  Xi  C  R^. 

The  trajectory  ri{t)  is  the  desired  output  trajectory,  and  yi{t)  is  the  output 
vector  which  shall  track  ri{t).  Notice  that  in  general  the  initial  set  may  be 
a  function  of  the  trajectory  thus  we  denote  it  as  Si{ri).  This  is  because 
even  though  trajectory  tracking  controllers  are  guaranteed  to  converge  for  any 
initial  condition,  trajectory  tracking  in  the  presence  of  state  constraints  or  input 
constraints  can  be  guaranteed  only  if  the  initial  tracking  error  is  sufficiently 
small.  In  this  paper  we  are  interested  in  switching  between  controllers,  rather 
than  the  design  of  output  tracking  controllers.  We  therefore  make  the  following 
assumption. 

Assumption  1  For  each  control  mode  qi,  i  G  iV},  we  assume  that  a 

controller  of  the  form  (2.2)  has  been  designed  which  achieves  output  tracking 
such  that  yi{t)  shall  track  ri{t)  where  ri  £  Hi  ^  0,  while  the  state  satisfies  the 
set  of  state  constraints  x(t)  £  Xi  C  R^,  when  the  initial  condition  of  the  system 
(2.1)  starts  in  the  set  Si{ri)  C  C  R’^. 

The  above  assumption  is  justified  given  the  maturity  of  output  tracking  con¬ 
trollers  for  large  classes  of  linear  and  nonlinear  systems  [15].  Based  on  different 
design  methodologies,  the  notion  of  output  tracking  could  be  different  as  it  could 
be  uniform  asymptotic,  exponential,  etc.  Depending  on  the  complexity  on  the 
computation,  one  may  choose  a  specific  notion  of  tracking  for  solving  Problem 
1.  In  order  to  motivate  the  discussion,  we  present  a  planar  helicopter  model 
and  a  set  of  controllers  in  which  each  controller  satisfies  Assumption  1  but  with 
different  output  functions  and  state  constraints. 

Example  1.  Multi-Modal  Control  of  a  Planar  Helicopter  Model.  In  this 
example,  a  helicopter  model  [4]  described  in  longitudinal  and  vertical  axes  with 
simplified  force  and  moment  generation  processes  is  considered.  The  x,  2-axes  of 
the  spatial  frame  are  pointing  to  north  and  down  directions.  The  body  x-axis 
is  defined  from  the  center  of  gravity  to  the  nose  of  the  helicopter,  and  body 
2-axis  is  pointing  down  from  the  center  of  gravity.  The  motion  of  the  helicopter 
is  controlled  by  main  rotor  thrust,  Tm  and  longitudinal  tilt  path  angle,  om-  The 
pitch  angle  is  defined  by  6.  The  equations  of  motion  can  be  expressed  as; 

cos  9{t)  sin  6{t)  -Tm  {t)  sin  a  M{t)  0  .23^ 

Pzh)  ”m  -sin6){t)  cos6{t)  _  -TM{t)  cos  a M{t)  \  [g\ 

9{t)  =  ~(MMaM(^)  +/imTm(0  sin  aM(t)) 


(2.4) 
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The  state  vector  and  input  vector  are  defined  as  x  =  \px,Px,Pz,Pz,0,6]'^  G 
and  u  =  respectively. 


Control  Mode 

Output 

Reference 

Constraint 

qi'.  Hover 

2/1  =  b>x,PzV' 

n 

92*  Cruise 

2/2  =  \Px,Pzf 

r2 

^2 

93:  Ascend 

2/3  ~ 

rs 

X3 

94:  Descend 

II 

^4 

X4 

Define  Xi  =  X2  R  x  xRx  {v„v^)  x  (-7r/2,7r/2)  x  M,  X3  =  R  x 

I'^x)  xRx  (VzyV^^)  X  {—7^ f2^7rf2)  xR,  and  X4  =  Rx  (v^^Vx)  xRx  x 

(-7r/2, 7r/2)  x  R  where  v^<0<vl^  <  and  <  0  <v^/  <v^.  To  sat¬ 

isfy  Assumption  1,  several  control  design  methodologies  can  be  used  to  design  a 
controller  for  each  discrete  control  mode  qi  where  i  G  {1, 2, 3, 4}.  Each  controller 
implementation  can  be  specified  as  u  =  ki(x,ri)  with  n  e  Hi  where  Hi  defines 
the  class  of  admissible  output  trajectories  in  mode  qi,  and  the  performance  of  the 
closed-loop  system  can  be  specified  by  initial  set,  Si(ri),  and  flow,  <j>i{t,ri,xo) 
where  xq  G  Si{ri). 

Given  two  control  modes,  one  cannot  simply  switch  from  one  control  mode 
to  another  due  to  incompatible  constraints.  A  natural  question  is  then  whether 
this  mode  reachability  task  can  be  achieved  by  a  finite  sequence  of  modes.  Based 
on  the  above  example,  we  can  now  define  the  mode  switching  problem  that  we 
will  address  in  this  paper. 

Problem  2  (Mode  Switching  Problem).  Given  an  initial  control  mode  qs  with 
desired  reference  rs,  does  there  exist  a  sequence  of  control  modes  such  that  the 
system  can  reach  a  desired  mode  qp  with  reference  rp?  If  so,  then  determine 
a  mode  sequence  qs  ^  ^  .  q^  qy  . .  qp  along  with  trajectories  ri  for  each 
control  mode  qi,  as  well  as  conditions  for  switching  between  the  control  modes. 

For  the  control  modes  defined  in  Example  1,  one  can  define  a  task  of  having 
the  Hover  mode  qi  as  an  initial  mode  and  ask  for  a  finite  control  mode  sequence 
to  reach  the  Ascend  mode  q^.  Any  solution  to  this  problem  leads  to  a  feasible 
execution  of  the  task  called  high- altitude  takeoff  according  to  flight  instruction  for 
helicopter  pilots.  Note  that  Problem  2  is  a  reachability  problem.  More  generally, 
one  can  envision  more  complicated  tasks  that  can  be  specified  in  temporal  logic, 
but  in  this  paper  we  restrict  our  attention  to  reachability  specifications. 


3  A  Mode  Switching  Condition 

In  its  full  generality.  Problem  2  can  be  posed  as  a  controller  synthesis  problem 
for  hybrid  systems  [7,12].  Such  synthesis  methods  involve  nested,  and  possibly 
cyclic  reachability  computations,  where  each  reachability  computation  involves 
computing  the  capture  set  of  a  differential  game.  Furthermore,  termination  guar¬ 
antees  for  controller  synthesis  methods  are  rather  limited  [6]. 
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In  our  mode  switching  problem,  however,  there  is  enough  structure  to  take 
advantage  of  in  order  to  simplify  the  complexity  of  the  synthesis  task.  First  of  all, 
the  continuous  controllers  are  assumed  to  have  been  designed,  and  therefore  we 
do  not  have  to  design  the  continuous  part  of  the  system,  but  simply  determine 
the  mode  switching  conditions.  Furthermore,  by  imposing  certain  conditions  on 
the  allowable  mode  switches,  we  reduce  the  complexity  of  the  synthesis  problem, 
by  maximally  decoupling  the  discrete  and  continuous  aspects  of  the  synthesis. 

To  address  the  problem,  we  have  to  characterize  the  reachable  set  of  each 
mode  and  switching  condition  among  them.  Let  (j)i{t,ri^XQ)  denote  the  flow  of 
system  (2.1)  operating  in  mode  qi  with  the  controller  defined  by  (2.2)  for  initial 
condition  xq,  and  desired  output  trajectory  Vi. 

Definition  2  (Predecessor  set).  Given  a  set  P  C  Xi,  a  trajectory  ri  €  Tli, 
the  reach  set  Prei{P,ri)  in  mode  qi  is  defined  by 

Prei{P,ri)  =  {  xq  e  Xi  |  >  0  G  P  such  that  x  =  <t>i{t,ri,XQ)  }  (3.1) 

Therefore  Prei{P,ri)  consists  of  all  states  that  can  reach  the  set  P  in  mode  qi 
for  a  given  output  trajectory  r^,  at  some  future  time.  Furthermore,  because  of 
Assumption  1,  we  have  a  guarantee  that  throughout  the  whole  trajectory,  the 
state  constraints  are  satisfied,  that  is  (l>i(t,ri,xo)  G  Xi  for  all  t>0. 

Given  control  modes  qi,  and  qj,  one  would  typically  allow  a  switch  from  mode 
qi  to  qj  if  during  the  operation  of  the  system  under  mode  qi  for  some  ri  ^  Pi, 
the  state  reaches  the  allowable  set  of  initial  conditions  Sj{rj)  for  some  Vj  G  Pj, 
i.e.  there  exist  ri  G  Pi  and  rj  G  Pj  such  that 


Siiri)nPrei{Sflrj),ri)^0,  (3.2) 

If  one  allows  this  type  of  mode  switching,  then  reachability  critically  depends  on 
the  particular  choice  of  initial  conditions  since  some  initial  conditions  in  Si{ri) 
may  reach  the  set  Sj{rj)  of  mode  qj  while  others  may  not.  If  this  is  the  case, 
then  nested  reachability  computations  seem  necessary  for  the  solution  of  the 
mode  switching  problem.  However,  such  nested  computations  can  be  avoided  if 
one  places  the  following  condition  on  mode  switching. 

Definition  3  (Consistent  mode  switching).  Assume  that  control  mode  qi 
satisfies  Assumption  1,  that  is  (t>i{t,ri,XQ)  G  Xi  for  all  t  >  0  with  initial  condi¬ 
tions  starting  from  Si{ri)  where  ri  ^  Pi.  A  transition  from  mode  qi  to  mode  qj 
is  allowed  only  if  there  exist  ri  G  Pi  and  rj  G  Pj  such  that 


Siiri)  C  Prei{Sj{rj),ri)  (3.3) 

^  ^xq  G  Si(ri)  >  0  G  Sj{rj)  such  that  x  =  4>i{t,ri,XQ)  (3.4) 

Therefore,  if  there  exist  trajectories  r,  (in  mode  qi)  and  Vj  (in  mode  qj)  such 
that,  if  the  system  starts  at  any  xq  G  Si{ri),  then  switching  from  mode  qi  to  qj 
can  occur  at  some  time  t  such  that  (l>i(t,ri,Xo)  6  Sj{rj).  The  consistent  mode 
switching  condition  is  shown  in  Figure  1.  The  condition  expressed  in  Definition  3 
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Fig.  1.  Visualization  of  consistent  mode  switching  condition 

is  a  consistency  condition  that  guarantees  that  our  ability  to  get  from  mode 
qi  to  mode  for  the  particular  trajectory  pair  {ri.Tj)  is  independent  of  the 
choice  of  initial  condition  in  Si{ri).  The  condition  is  reminiscent  of  the  time- 
abstract  bisimulation  property  from  formal  verification  [8].  In  this  case,  however, 
Definition  3  is  quite  different  since  no  partitioning  of  the  state  space  is  involved. 
Now  define 


7^*^  —  6  X  'Jtj  I  condition  (3.3)  is  satisfied  }  (3.5) 

Hence,  if  0,  then  mode  switching  from  qi  to  qj  is  possible  since  there  exists 
a  trajectory  €  TZi  that  will  steer  the  system  state  to  an  initial  set  Sj{rj)  with 
Tj  E  IZj  independently  of  where  we  start  in  Siiri).  Therefore,  every  trajectory 
pair  {ri.Tj)  e  will  steer  the  system  from  mode  q^  to  mode  q^.  For  each 
{ri.Tj)  E  7^*^,  the  only  thing  that  depends  on  the  initial  condition  is  when  the 
state  will  reach  but  not  if  the  state  will  reach  Sj(rj). 

To  test  the  mode  switching  condition  (3.3),  and  compute  the  sets  ,  one 
needs  to  compute  the  predecessor  set  Prei{P,ri).  Even  though  there  is  exten¬ 
sive  research  in  computing  exactly,  or  approximately  such  reachable  sets  [7,9,12, 
11,13],  there  is  limited  research  for  parametric  reachability  computations  [10]. 
Furthermore,  in  our  problem  we  take  advantage  of  the  fact  that  in  each  control 
mode,  the  output  is  tracking  a  reference  trajectory  r^.  Therefore,  by  designing 
trajectories  we  design  part  of  the  reachable  space  whereas  the  part  of  the  state 
is  not  reflected  in  the  output  remains  within  the  set  Xi.  Choosing  simple,  or 
better  computable,  classes  of  trajectories  TZi  will  allow  us  to  efficiently  perform 
reachability  computations  for  Prei{P,ri)  with  parameters  ri  e  TZi.  To  continue 
discussion,  we  assume  that  the  Prci  operators  are  available  to  us,  and  defer  this 
important  issue  to  Section  5. 

4  Mode  Sequence  Synthesis 

The  mode  switching  condition  (3.3)  makes  the  mode  switching  problem  much 
more  tractable  since  we  can  ignore  the  initial  sets  and  focus  on  the  trajectory 
sets  1Z^f  Furthermore,  the  construction  presented  in  this  section  will  abstract 
the  mode  switching  logic  into  a  purely  discrete  graph.  Therefore  one  can  first 
determine  the  sequence  of  modes  using  standard  algorithms  for  discrete  graph 
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reachability,  and  then  determine  the  continuous  parameters  ri  for  each  mode. 
This  will  decouple  the  discrete  from  the  continuous  aspects  of  the  problem,  and 
allow  continuous  techniques  for  continuous  problems,  and  discrete  techniques  for 
discrete  problems. 

Given  a  collection  of  control  modes  Q  =  {g^i, . . . ,  g^v},  the  first  attempt  at 
solving  the  mode  switching  would  construct  a  graph  as  (Q,  -^)  where  the  vertices 
of  the  graph  would  be  the  set  of  control  modes  Q,  and  we  would  define  the 
transition  relation  Q  x  Q  eis 


{Qu(lj)  0  (4.1) 

In  other  words,  there  would  be  a  transition  qi  — >  if  there  exist  trajectory  pairs 
(ri,rj)  G  that  can  transfer  the  system  from  mode  qi  to  qj.  This  approach, 
however,  leads  immediately  to  problems  because  if  qi  qj  and  qj  ->  qk  there 
may  not  exists  a  trajectory  r^,  which  will  take  a  point  x  6  Si{ri)  to  Sk{rk) 
via  Sj{rj),  if  =  0.  Hence,  transitivity  fails,  and  our  mode  switching 

graph  is  not  a  consistent  abstraction  as  the  high  level  mode  switching  logic  is 
not  implementable  at  the  lower  level  by  the  continuous  controllers. 

In  order  to  obtain  a  consistent  control  mode  graphs  denoted  as  {Qc,  -^c),  that 
has  feasible  low  level  implementations,  our  original  attempt  must  be  refined.  In 
particular,  each  control  mode  qi  gets  refined  to  2N  submodes,  where  N  submodes 
stand  for  entering  mode  qi  from  any  other  mode  qj,  and  N  more  copies  for  exiting 
mode  qi  towards  any  other  mode  qj.  This  refinement  is  illustrated  in  Figure  2, 
where  mode  qi  has  two  submodes,  q\^  which  is  the  operation  of  the  system  in 
mode  qi  on  the  way  to  mode  q2,  whereas  is  the  operation  of  the  system  under 
mode  qi  after  being  in  mode  q2.  Therefore,  this  control  mode  graph  has  some 
discrete  memory,  in  the  sense  that  each  state  represents  not  only  which  mode 
the  system  is  in,  but  also  which  mode  will  either  precede  it  or  has  preceded  it. 


Fig.  2.  Refining  the  mode  switching  logic  by  introducing  submodes  in  order  to  obtain 
a  consistent  control  mode  graph 

The  pairwise  reachability  computations  in  order  to  compute  the  sets  , 
can  immediately  be  embedded  in  the  graph  The  computed  sets 

can  be  used  to  go  from  submode  q^  to  qJ .  After  this  initial  step,  the  graph 
contains  only  isolated  transition  pairs  between  different  modes  as  no  transitions 
between  submodes  are  considered. 

If  the  set  can  be  expressed  as  a  decoupled  product  of  the  form  = 
TVP  X  TlV  where  =  {r^  G  Hi  \  {ri,rj)  G  and  TVf  =  {rj  G  Hj  |  (ri,rj)  G 
then  the  choice  of  trajectory  ri  G  in  mode  qi  would  work  for  any 


340  T.J.  Koo,  G.J.  Pappas,  and  S.  Sastry 


trajectory  Vj  e  TVJ'  in  mode  i.e. 


Vr^  G  Wrj  e  TlJ  condition  (3.3)  is  satisfied. 


This  decoupling  allows  us  to  consider  switching  via  submodes.  In  Figure  2,  if 
7^2^  n  7^2^  is  non  empty,  then  that  means  that  there  exists  a  trajectory  r2  which 
is  common  for  both  submodes.  Notice  that  in  this  case,  we  do  not  have  to  do  any 
reachability  computations,  we  simply  have  to  compute  intersections  of  trajectory 
sets.  Therefore,  within  each  mode,  we  can  check  for  submode  consistency  by  sim¬ 
ply  performing  set  intersections.  Since  there  are  maximally  2N  submodes  of  N 
modes,  a  total  of  N{N)'^  —  intesections  must  be  computed.  We  now  summa¬ 
rize  the  ideas  and  present  an  algorithm  for  constructing  the  consistent  control 
mode  garph.  The  algorithm  starts  with  the  pairwise  reachability  computations 
(3. 3, 3. 5),  and  performs  the  submode  interconnections. 


Algorithm  1  :  (Consistent  Control  Mode  Graph) 
Input  Control  Modes  Q  =  {gi, . . . , 

Output  Control  Mode  Graph  (Qc,  ~>c) 

Initialize  Qc  :=  0,  ~^c=  0 

Determine  Mode  Interconnections 
for  i  =  l  :  N 
for  j  =  1  :  N 

Compute  sets  7^"^  using  (3.3)  and  (3.5) 

\fn^^  =  n^  x7^y; 

9?  —  9h  of  '=  9j, 

Qc  := 

U{{qr,qf)} 

end  if 
end  for 
end  for 

Determine  Submode  Interconnections 
for  j  =  1  :  N 

for  all  qf  S  {q^  e  Q,|3n  {q^\qf)  £->c} 

for  all  gf  €  {9^’"  e  Qe|3m  s.t. 
if7^y  n7^f  5^0; 

end  if 
end  for 
end  for 
end  for 
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Overall,  Algorithm  1  requires  reachability  computations  for  the  mode 
interconnections,  and  set  intersections  for  the  submode  interconnections. 
After  applying  Algorithm  1,  we  obtain  a  finite  control  mode  graph  (Qo-^c) 
which,  as  the  following  proposition  shows,  is  consistent. 

Proposition  1.  For  any  j  G  if  3^^  ^  G  Qc\  3n  such  that 

€^c},  Bqf  e  {^r  e  Qc\  3m  such  that  (gf  ,9^)  e-^c}  and 
TV^  n  7^  0,  then  there  exists  ri  S  TT/  ,  Vj  €  TC?  fl  TV^  and  rk  €  such 
that 

Si{ri)  C  Prei{Prej{Sk,rk),rj). 

Proof:  Given  {ql^,qj^)  G-^c,  we  can  pick  any  ri  G  Pf'i  and  since  /  0  we 

can  pick  any  G  fl  so  that  Vxq  G  Si{ri)  3t  >  0  3x  G  Sj{rj)  such  that 

X  =  ^i{t,ri,XQ).  Then,  pick  any  rk  G  ,  since  e->c,  €  Uj  nUf 

and  the  switching  occurs  whenever  (j)i{t.,ri^XQ)  G  Sj{rj).,  it  can  be  easily  seen 
that  3s  >  0  3y  e  Sk{rk)  such  that  y  =  =  (j)j{s,rj,x). 

The  choice  on  the  trajectories  is  illustrated  in  Figure  3.  Since  by  Assumption  1, 
‘j  •)  €  *5  *)  ^  for  the  choice  of  initial  conditions  and  reference 

trajectories,  by  directly  applying  the  definition  we  have  shown  the  result.  □ 


Fig.  3.  Graphical  illustration  of  feasible  trajectories  between  control  modes. 

Without  loss  of  generality,  in  the  following  discussion,  we  assume  that  the 
given  initial  and  final  control  mode  in  Q  can  be  represented  by  qs  ^  Qc  and 
qp  ^  Qc  respectively.  Given  an  initial  control  mode  qs  E  Qc,  the  problem  of 
whether  we  can  reach  control  mode  qp  G  Qc,  can  be  efficiently  solved  using 
standard  reachability  algorithms.  Furthermore,  one  can  determine  the  shortest 
path  (minimum  number  of  mode  switches)  between  mode  qs  and  qp^  in  the  con¬ 
trol  mode  graph.  The  structure  that  we  have  imposed  on  our  control  mode  graph, 
immediately  results  in  the  following  solution  to  the  mode  switching  problem. 

Theorem  1  (Mode  Switching  Solution).  Given  a  collection  of  control  modes 
Q,  consider  the  mode  switching  Problem  2.  Construct  the  consistent  control  mode 
graph  (Qc,  -^c)  cls  described  in  Algorithm  1.  If  there  exists  a  path  in  the  consistent 
control  mode  graph  between  qs  and  qp  with  feasible  trajectories  rs  and  rp,  then 
Problem  2  is  solvable. 

Having  determined  the  sequence  of  modes  that  can  steer  our  system  from 
qs  to  qp,  we  are  left  with  the  problem  of  determining  the  parameters  for 
each  mode  of  the  sequence.  By  construction,  such  parameters  exist  and  may 
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be  selected  from  the  computed  sets.  Furthermore,  it  is  reasonable  to  pose  the 
problem  of  choosing  within  mode  qi  as  an  optimization  or  an  optimal  control 
problem.  A  key  issue  for  this  approach  (as  well  as  for  most  controller  synthesis 
approaches  for  hybrid  systems),  is  to  be  able  to  compute  Prei(Sj^ri)  in  order 
to  check  condition  3.  This  is  the  focus  of  the  following  section  of  this  paper. 

5  Reachability  Computations 

There  has  been  a  growing  interest  recently  in  computing  reachable  sets  for  var¬ 
ious  classes  of  systems  [9,11,7,13].  In  particular,  the  approach  of  [9]  has  been 
extended  to  classes  of  parametric  linear  control  systems  [10],  which  is  highly 
relevant  for  computing  the  operator  (3.1). 

In  our  case,  however,  the  continuous  dynamics  are  those  of  out  put- tracking, 
closed-loop  systems.  Therefore  part  of  the  state  is  forced  to  converge  to  a  tra¬ 
jectory  that  we  get  to  design,  and  part  of  the  trajectory  is  guaranteed  to  satisfy 
state  constraints.  This  gives  us  the  opportunity  to  obtain  very  reasonable  approx¬ 
imations  of  the  reachable  sets,  and  even  design  reachable  sets  by  appropriately 
designing  output  trajectories.  The  following  example  illustrates  how  continuous 
controller  design  results  in  reachability  computations  which  are  very  easy  to 
check. 

Example  2.  Multi-Modal  Control  of  a  Helicopter  Model(Continued)  Re¬ 
consider  the  four  control  modes  shown  in  Example  1.  We  first  present  the  con¬ 
troller  design  to  illustrate  how  to  compute  the  reachable  sets,  then  we  show 
how  to  check  the  consistent  mode  switching  condition  between  control  modes. 
In  this  example,  we  assume  that  all  output  trajectories  are  constant  trajectories, 
therefore,  all  controllers  are  setpoint  regulators.  Choosing  computable  classes  of 
trajectories  makes  the  reachability  computations  simpler. 

Given  the  specifications  for  the  control  modes,  a  nonlinear  control  scheme 
[5]  based  on  outer  flatness  is  applied  for  the  design  of  the  controllers. 
For  each  mode,  the  closed-loop  dynamics  with  states  defined  by  Xex  — 
\Px,PxtPztPz,^,^tTm^o,m]'^  ^  can  be  decoupled  into  an  inner  system  and 
two  outer  subsystems  which  specify  the  dynamics  in  x  and  z  directions.  In  the 
following  presentation,  the  Hover  mode  is  presented  to  illustrate  how  the  reach¬ 
able  set  can  be  computed. 

For  qi ,  the  output  tracking  controller  is  designed  such  that  yi  (t)  shall  track 
’’i  =  [f'lx^f'iz]'^  and  the  output  tracking  error  is  uniformly  ultimately  bounded. 
Furthermore,  because  of  satisfying  Assumption  1,  the  controller  is  designed  with 
initial  set  5i(ri)  =  5H[ni,0]^,eia,)xB([ri^,0]^,ei^)x5i„  wherer,  e  7^i  = 

>  0  and  Sin  Q  (~^/2,7r/2)  x  R®  such  that  for  x(fo)  6  ‘S'i(ri)  then 

’  ||eix(f)||  <  Mix  exp(-Q:i^t)(||eix(fo)||  +  5ii„x),  f  ||eix(t)ll  <  <5ix, 
lieuWII  <  Mi^exp(-aixt)(||eix(to)||  +  ^linz)j  and  I  iieu(t)||  <  Su, 

^in  €  ^  t  <.  to  Ti]  y  Xin  G  Sin,  Vt  >  to  +  Ti 

^ _  (5.1) 

B{r,e)  =  {77I  \\T)-r\\  <  e}. 
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for  SOm©  T\  ^12 ?  ^linz^  ^l2  ^  0*  Itl  3,b0V6,  ©lx  — 

[Px  -  rix,Px]'^,  eu  =  \Pz-  riz.Pz]'^,  and  Xin  =  [Oj,TM,aM]'^-  Equation  (5,1) 
explicitly  over  specifies  the  reachable  set  of  the  mode  qi  by  examing  the  stability 
property.  For  other  modes,  although  the  control  designs  are  slightly  modified 
for  tracking  different  outputs,  the  reachable  sets  of  other  modes  are  similarly 
computed.  In  Figure  4,  we  show  the  inital  sets  of  all  the  control  modes  by 


Fig.  4.  Projection  of  5i(ri),  52(r2),  S'3(r3),  and  54(7-4)  onto:  (a)  px  —  plane;  (b) 
Pz  —  Pz  plane 


projecting  them  onto  Px  ~  Px  plane  and  p^  —  Pz  plane  where  the  projection 
operator  is  defined  as  Ili  :  Xex  ^  (Pi^Pi)  for  i  ^  In  summary,  the  control 

modes  can  be  specified  by 


Control  Mode 

Trajectory  Set 

Initial  Set 

Qi 

Til  =  R'-" 

Si  =  5(0,4)  X  5(0,4)  X  Sin 

Q2 

7^2  =  [-3,3]  X  {0} 

52  =  K  X  (-3.5, 3.5)  X  (-3.5, 3.5)  x  Si„ 

Qs 

53  =  R  X  (1.5, 4.5)  X  (-3.5, 0.5)  x  5i„ 

Q4 

■Ri  =  [2, 4]  X  [0, 3] 

54  =  R  X  (1.5, 4.5)  X  (-0.5, 3.5)  x  Sin 

where  Sin  =  B(0, 0.2),  Xin  ~  {“7r/2,7r/2)  x  and  the  associated  parameters 
are  defined  as  =  —6,  ^l,Vx  =  6,  =  —6,  =  1,  =  —1,  Vz  =  b- 

Given  the  set  of  control  modes,  we  generated  the  consistent  control  mode 
graph  by  applying  Algorithm  1.  In  Figure  5,  we  illustrate  the  idea  of  computing 
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the  reachable  sets  on  —  px  plan.  One  can  easily  see  the  advantage  of  using 
feedback,  since  it  is  straight  forward  not  only  to  check  the  consistent  mode 
switching  condition  but  also  to  determine  the  feasible  range  of  trajectory,  that 
is  compute  the  sets  TVK  In  particular  consider  the  pair  (^1,^2)?  that  is  the 
transition  from  hover  to  cruise.  As  can  be  seen  from  the  left  side  of  Figure  5,  the 
consistency  condition  is  trivially  satisfied  since  the  ball  Si{ri)  will  eventually 
shrink  towards  the  setpoint  {ria;,0),  and  as  a  result,  will  be  totally  contained 
inside  S2{r2)  for  any  r2.  Therefore,  in  this  case  =  1Zi  x  7^2-  Therefore, 


Fig.  5.  Graphical  illustration  of  performing  reachability  computation  for  checking  con¬ 
sistent  mode  switching  condition  on  px  —  Px  plane:  (a)  92;  (b)  q2  ->  qs 

feedback  allows  us  to  check  very  easily  the  consistency  condition  and  compute 
the  sets  .  The  right  side  of  Figure  5  shows  the  similar  graphical  computation 
for  the  mode  transition  (925^3))  from  cruise  to  ascend.  In  a  similar  manner,  we 
have  checked  the  following  pairs, 

{(gi,g2),fe,92),(g2,93),te,g4),fe,g2),(93,g3),(g3,g4),(g4,^f2),(94,g3),(g4,g4)} 

All  of  the  above  reachability  computations  were  extremely  simple  to  check.  The 
result  of  applying  Algorithm  1  is  summarized  in  the  control  mode  graph  that  is 
shown  in  Figure  2. 


Fig.  6.  Consistent  control  mode  graph  for  the  multi-modal  helicopter  control  example 

Recall  the  high- altitude  takeoff  task,  which  is  the  task  of  having  the  Hover 
mode  qi  as  an  initial  mode  and  ask  for  a  finite  control  mode  sequence  to  reach 
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the  Ascend  mode  q^.  We  can  now  see  that  from  Figure  2  that  qi  has  {q\^}  as 
a  submode,  and  qz  =  {ql^ ,  qi^ ,  ql^ ,  ql^} ,  and  there  exit  many  paths  which  are 
feasible  for  achieving  the  task.  However,  ql^ql^ql^ql^  gives  a  solution  to  the 
task  with  the  minimum  number  of  mode  switches,  ie.,  qi  ->  q2  ^  qs.  Given  a 
cost  function  with  respect  to  the  continuous  variables,  the  performance  of  the 
sequence  can  now  be  optimized  with  respect  to  the  feasible  trajectories.  We  have 
therefore  decoupled  the  problem  in  a  purely  discrete  graph  search  problem,  and 
a  collection  of  continuous  designs  within  each  mode. 

Simulation  results  of  the  controlled  system  based  on  the  selected  sequence 
are  shown  in  Figure  2.  In  the  simulation,  we  can  choose  ri  =  [0  0]^  G 
r2  =  [2  0]^  G  n  and  ra  =  [3  —  1]^  G  The  initial  conditions  of  the 
outer  system  are  p:,(0)  =  -2,  p^(0)  =  -0.2,  p^(0)  =  1,  p^O)  =  0.5.  The  initial 
condition  of  the  inner  system,  a:in(0)  G  Sin.  Mode  switchings  occur  at  t  =  20  for 
-)■  q2  and  at  t  =  45  for  q2  ->  93. 


Fig.  7.  Projected  trajectories  of  the  helicopter  along  with  the  initial  sets  of  the  next 
control  modes  from  different  view  angles  are  shown.  Notice  that  immediate  transition 
q2  —>  qs  after  qi  — >■  q2  is  not  allowed  until  x{t)  enters  the  initial  set  53(r3). 


6  Conclusion 

In  this  paper,  we  have  considered  the  mode  switching  problem  among  a  collection 
of  output  tracking  controllers  for  nonlinear  systems.  Our  approach  consists  of 
extracting  a  finite  graph  which  refines  the  original  collections  of  modes,  but  is 
consistent  with  the  physical  system.  Extracting  a  finite  graph  critically  depends 
on  the  fact  the  closed  loop,  output  tracking  controllers  reduce  the  complexity  of 
the  model  to  the  complexity  of  the  output  trajectories. 

Even  though,  our  framework  reduces  the  continuous  complexity  so  that  many 
of  the  computations  can  be  done  by  hand,  obtaining  a  consistent  mode  graph 
for  a  large  scale  helicopter  or  aircraft  (a  Boeing  747  has  approximately  500 
modes)  will  clearly  require  the  development  of  a  computational  tool.  Such  a 
mode  switching  tool  can  be  used  off-line  for  synthesizing  the  mode  switching 
logic  every  time  a  new  mode  is  designed.  The  control  mode  graph  can  then  be 
used  on-line  for  efficient  and  dependable  real-time  mode  switching. 
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Abstract.  In  this  paper,  the  stability  of  switched  linear  systems  is  inves¬ 
tigated  using  piecewise  linear  Lyapunov  functions.  Given  a  switched  lin¬ 
ear  system,  we  present  a  systematic  methodology  for  computing  switch¬ 
ing  laws  that  guarantee  stability  based  on  the  matrices  of  the  system. 
We  assume  that  each  individual  subsystem  is  stable  and  admits  a  piece- 
wise  linear  Lyapunov  function.  Based  on  these  Lyapunov  functions,  we 
compose  “global”  Lyapunov  functions  that  guarantee  stability  of  the 
switched  linear  system.  A  large  class  of  stabilizing  switching  sequences 
for  switched  linear  systems  is  characterized  by  computing  conic  parti¬ 
tions  of  the  state  space.  The  approach  is  applied  to  both  discrete-time 
and  continuous- time  switched  linear  systems. 


1  Introduction 

In  this  paper,  we  study  the  stability  of  continuous  and  discrete-time  switched 
linear  systems  using  piecewise  linear  Lyapunov  functions  and  we  identify  classes 
of  switching  sequences  that  result  in  stable  trajectories.  The  main  motivation 
behind  this  problem  is  that  it  is  often  easier  to  find  switching  controllers  than  to 
find  a  fixed  controller.  In  the  case  when  we  have  multiple  control  objectives,  we 
may  design  a  continuous  controller  for  each  control  objective,  and  control  the 
behavior  of  the  plant  by  switching  between  different  controllers.  For  example, 
in  the  control  of  the  longitudinal  dynamics  of  an  aircraft  with  constrained  angle 
of  attack,  the  control  objective  is  twofold:  track  the  pilot’s  reference  normal 
acceleration  while  maintaining  a  safety  constraint  in  the  angle  of  attack  [8].  A 
continuous  feedback  control  law  can  be  easily  designed  for  each  control  objective 
resulting  in  two  asymptotically  stable  subsystems  and  a  switching  mechanism 
can  be  used  to  simultaneously  achieve  both  objectives.  Such  a  switching  system 
might  become  unstable  for  certain  switching  sequences,  even  if  all  the  individual 
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the  Army  Research  Office  (DAAG55-98-1-0199)  is  gratefully  acknowledged. 

M.D.  Di  Benedetto,  A.  Sangiovanni-Vincentelli  (Eds.):  HSCC  2001,  LNCS  2034,  pp.  347-360,  2001. 
©  Springer- Verlag  Berlin  Heidelberg  2001 


348  X.D.  Koutsoukos  and  P.J.  Antsaklis 


subsystem  are  stable  (see  for  example  [8]),  For  such  problems,  it  is  important  to 
characterize  switching  sequences  that  result  in  stable  trajectories. 

Stability  of  switched  systems  has  been  studied  extensively  in  the  literature; 
see  for  example  [8,16,17]  and  the  references  therein.  Sufficient  conditions  for  uni¬ 
form  stability,  uniform  asymptotic  stability,  exponential  stability  and  instability 
were  established  in  [22].  Necessary  conditions  (converse  theorems)  for  some  of 
the  above  stability  results  have  also  been  established.  Analysis  tools  for  switched 
and  hybrid  systems  based  on  multiple  Lyapunov  functions  were  presented  in  [5]. 
Stability  analysis  of  switched  systems  is  usually  carried  out  using  a  Lyapunov-like 
function  for  each  subsystem  [8].  These  Lyapunov  functions  are  pieced  together 
in  some  manner  in  order  to  compose  a  Lyapunov  function  that  guarantees  that 
the  energy  of  the  overall  system  decreases  to  zero  along  the  state  trajectories 
of  the  system.  The  application  of  the  theoretical  results  to  practical  hybrid  sys¬ 
tems  is  accomplished  usually  using  a  linear  matrix  inequality  (LMI)  problem 
formulation  for  constructing  a  set  of  quadratic  Lyapunov-like  functions  [12,21]. 
Existence  of  a  solution  to  the  LMI  problem  guarantees  that  the  hybrid  system  is 
stable.  However,  in  order  to  formulate  the  LMI  problem,  a  partition  of  the  state 
space  and  therefore  a  switching  law  must  be  known  a  priori.  Usually,  such  a 
partition  consists  of  a  set  of  ellipsoidal  regions  derived  by  exploiting  the  physical 
insight  for  the  particular  application.  Although,  the  LMI  approach  for  hybrid 
system  stability  is  computationally  efficient,  it  is  based  only  on  sufficient  con¬ 
ditions  and  more  importantly,  it  relies  on  a  particular  partition  chosen  by  the 
designer. 

In  order  to  investigate  the  stability  properties  of  practical  hybrid  systems, 
there  is  an  important  need  to  characterize  partitions  of  the  state  space  that 
lead  to  stable  trajectories  based  on  the  system  parameters.  Such  partitions  can 
be  used  very  efficiently  for  the  design  of  switching  control  laws  that  guarantee 
stability  of  the  overall  system.  In  our  approach,  we  characterize  a  large  class 
of  switching  sequences  that  result  in  stable  trajectories.  Given  a  switched  lin¬ 
ear  system,  we  present  a  systematic  methodology  for  computing  switching  laws 
based  on  the  system  parameters  that  guarantee  stability.  We  assume  that  each 
individual  subsystem  is  stable  and  admits  a  piecewise  linear  Lyapunov  function. 
Based  on  these  Lyapunov  functions,  we  compose  “global”  Lyapunov  functions 
that  guarantee  stability  of  the  switched  linear  system.  The  main  contribution  of 
this  work  is  that  based  on  the  piecewise  linear  Lyapunov  functions  we  construct 
a  conic  partition  of  the  state  space  that  is  used  to  characterize  a  large  class  of 
switching  laws  that  result  in  stable  trajectories. 

It  should  be  noted  that  the  problem  considered  in  this  paper  has  been 
addressed  using  multiple  Lyapunov  function  tools  under  the  assumption  that 
switching  among  stable  systems  is  slow  enough  [8,16].  Here,  we  consider  piecewise 
linear  Lyapunov  functions  and  we  develop  a  systematic  approach  to  character¬ 
ize  stabilizing  switching  sequence  that  offers  a  significant  advantage.  Individual 
piecewise  linear  Lyapunov  functions  are  “pieced  together”  in  a  systematic  way 
and  they  result  in  a  conic  partition  of  the  state  space  that  can  be  used  very 
efficiently  for  the  design  of  the  switching  control  law.  Note  that  the  paper  re- 
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ports  results  from  [14]  and  that  early  results  for  the  discrete-time  case  have  been 
reported  in  [15], 

The  paper  is  organized  as  follows.  In  Section  2,  the  problem  of  identifying 
stabilizing  switching  sequences  is  described.  Section  3  presents  the  necessary 
background  for  piecewise  linear  Lyapunov  functions.  The  emphasis  is  put  on 
computational  methods  for  constructing  such  Lyapunov  functions.  The  technical 
results  for  the  characterization  of  stabilizing  switching  sequences  are  presented  in 
Section  4.  The  application  of  the  methodology  to  continuous-time  switched  linear 
systems  is  presented  in  Section  5.  Finally,  concluding  remarks  are  presented  in 
Section  6. 


2  Problem  Statement 

In  this  section,  we  consider  discrete-time  switched  linear  systems  described  by 

x{t  +  l)  =  AgX{t),  q  eQ  =  {!,..  ^,N}  (1) 

where  x(t)  €  e  Z+  (the  set  of  nonnegative  integers)  and  Ag  e 

The  mathematical  model  described  by  (1)  represents  the  continuous  (state) 
portion  of  a  piecewise  linear  hybrid  dynamical  system.  The  particular  mode  q 
at  any  given  time  instant  may  be  selected  by  a  decision-making  process.  In  this 
paper,  we  represent  such  a  decision-making  process  by  a  switching  law  of  the 
form 

q{t  +  l)  =  6{q{t),x{t)).  (2) 

Given  x{t),  the  next  state  is  computed  using  the  mode  q{t),  that  is  x{t  +  1)  = 
Ag(t)^(t).  The  function  ^  :  Q  x  ->  51?’^  is  discontinuous  with  respect  to  a::.  A 
switching  law  is  defined  here  using  a  partition  of  the  state  space. 

Our  objective  is  to  investigate  the  stability  of  the  switched  linear  system  (1) 
under  the  switching  law  (2).  Note  that  the  origin  =  0  is  an  equilibrium  for 
the  system  (1).  Furthermore,  for  a  particular  switching  law,  the  switched  system 
(1)  can  be  viewed  as  a  special  case  of  a  time- varying  linear  system,  and  therefore 
the  usual  definitions  of  stability  can  be  used;  see  for  example  [1]. 


3  Piecewise  Linear  Lyapunov  Functions 

In  this  section,  we  briefly  present  some  background  material  necessary  for  the 
stability  analysis  of  switched  linear  systems  presented  later  in  this  paper.  We 
consider  the  discrete-time  linear  system 

x{t  +  1)  =  Ax{t)  (3) 

where  x{t)  G  and  A  G 

Definition  1.  A  nonempty  set  P  C  is  said  to  be  (^positivelyj  invariant  for 
the  system  (3)  if  x(0)  €  P  implies  that  x{t)  G  P  for  every  t  G  (Z+)  Z. 
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In  the  case  when  the  system  admits  a  positively  invariant  polyhedral  set 
P  containing  the  origin  a  Lyapunov  function  can  be  constructed  by  consider¬ 
ing  the  Minkowski  functional  {gauge  function)  of  P;  see  for  example  [3].  For 
bounded  invariant  polyhedral  sets  this  is  accomplished  as  follows  (the  extension 
to  unbounded  polyhedral  sets  is  straightforward): 

Let  Fj  be  a  face  of  a  polytope  and  consider  the  corresponding  hyperplane  Hi 
as  shown  in  Fig.  1.  The  hyperplane  can  be  described  (perhaps  after  normaliza¬ 
tion)  by  Hi  =  {x  ^  :  {x,Wi)  =  1}.  where  Wi  E  5?^  is  the  gradient  vector  of 

the  hyper  plane  and  (•,  •)  denotes  the  inner  product. 


Fig.  1.  A  poly  tope  P,  a  face  Fi  and  its  corresponding  hyperplane  Hi. 


Since  the  set  P  includes  an  open  neighborhood  of  the  origin,  can  be 
partitioned  into  a  finite  number  of  cones  defined  as  follows.  Each  face  F  of  the 
pol)d;ope  can  be  described  as  the  convex  hull  of  its  extreme  points  fj  €  5?"^,  j  = 
1, . . .  ,r.  A  finitely  generated  cone  can  be  defined  for  the  face  F  by 

r 

cone{F)  =  {a:  e  SR"  :  i  aj  >0,  j  =  l . r}.  (4) 

j-1 

Consider  a  polytope  P  c  and  assume  that  0  e  int(P).  The  Minkowski 
functional  of  P  is  defined  by 

V (x)  =  inf{p  >  0\x  e  pP}  (5) 

where  pP  ~  {px\x  €  P}.  Consider  a  particular  face  Fi  and  the  corresponding 
cone.  Since  Fi  G  dP  there  exist  unique  p  >  0  and  x  e  Fi  such  that  for  any 
X  e  cone  (Pi)  we  have  x  =  px  and  the  Minkowski  functional  can  be  computed  by 

=  ||||j^  =  P  =  p{x,  Wi)  =  (x,  Wi)  (6) 

since  (x,Wi)  =  1.  Therefore,  for  x  G  cone(Pi),  the  Lyapunov  function  induced 
by  the  set  P  can  be  written  as  V{x)  ~  {x,Wi).  Consequently,  the  Lyapunov 
function  induced  by  P  can  be  computed  for  x  G  by 

y{x)  =  max  ix.Wi) 


(7) 
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A  special  case  of  piecewise  linear  Lyapunov  functions  arise  when  the  posi¬ 
tively  invariant  set  P  of  Definition  1  is  centrally  symmetric.  In  this  case,  the 
Lyapunov  function  V{x)  can  be  represented  using  the  infinity  norm.  Further¬ 
more,  there  exists  a  class  of  linear  systems  for  which  such  a  Lyapunov  function 
can  be  computed  very  efficiently.  Consider  the  following  Lyapunov  function  can¬ 
didate  V{x)  ==  \\Wx\\oo  where  W  G  and  |1  •  \\oo  denotes  the  infinity  norm 

defined  by  lla:||oo  =  maxi<i<n 

Theorem  1.  [2]  V{x)  =  \\Wx\\oo  is  a  Lyapunov  function  for  the  system  (3) 

if  and  only  if  there  exist  a  matrix  Q  G  such  that  W A  —  QW  =  0  and 

||Q||oo<l. 

It  should  be  noted  that  similar  results  have  been  established  for  differential 
and  difference  inclusions  in  [19]. 


3.1  Computation  of  Piecewise  Linear  Lyapunov  Functions 

In  order  to  study  the  stability  properties  of  the  switched  linear  system  (1)  we 
assume  that  each  individual  subsystem  admits  such  a  piecewise  linear  Lyapunov 
function.  The  efficient  computation  of  each  Lyapunov  function  is  very  important 
for  the  application  of  the  proposed  methodology  to  practical  hybrid  systems.  In 
the  previous  section,  we  described  a  class  of  piecewise  linear  functions  induced  by 
polyhedral  sets  that  contain  the  origin.  A  Lyapunov  function  for  each  individual 
subsystem  can  be  defined  by  computing  a  positively  invariant  polyhedral  set  for 
the  subsystem.  In  the  following,  we  briefly  give  the  necessary  background  for 
the  computation  of  these  piecewise  linear  Lyapunov  functions.  First,  we  briefly 
describe  an  important  class  of  systems  for  which  positively  invariant  polyhedral 
sets  and  the  corresponding  Lyapunov  functions  can  be  computed  by  a  similarity 
transformation  [2].  In  this  case,  the  Lyapunov  functions  can  be  described  using 
the  infinity  norm.  Second,  we  outline  an  algorithm  [6,7]  which  can  be  used  for 
the  computation  of  general  positively  invariant  polyhedral  sets. 

A  class  of  linear  systems  for  which  such  a  Lyapunov  function  can  be  computed 
very  efficiently  is  presented  in  [2].  Consider  the  system  rr(t-f  1)  =  Ax{t)  where  the 
eigenvalues  of  the  matrix  A  are  located  in  the  complex  plane  within  the  square 
defined  by  the  vertices  (1,0),  (0,  i),  (-1, 0),  and  (0,  -z).  Then,  the  following  result 
is  shown. 

Corollary  1.  [2].  If  all  the  eigenvalues  Aj  =  /Zj  ±  cr^  of  the  order  linear 

system  x{t  +  1)  =  Ax{t)  are  in  the  open  square  |/Zi|  +  |(Ji|  <  1,  then  there  exists 
a  matrix  W  G  with  rankW  =  n  such  that  the  polyhedral  set  P  —  {x  G^^  : 

IjlFxIloo  <  1}  is  0,  positively  invariant  set  for  the  system. 

The  matrix  W  can  be  computed  as  the  solution  to  the  matrix  equation 


WA-QW  =  0  (8) 

with  the  condition  ||Q2:||oo  <  1-  It  is  well  known  [10]  that  if  the  matrices  A  and 
Q  do  not  have  common  eigenvalues  then  (8)  has  only  the  trivial  solution  W  —  0. 
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The  important  assumption  in  Corollary  1  is  that  W  e  with  rankkP  =  n.  In 
this  case,  W  can  be  computed  as  the  similarity  transformation  matrix  by  which 
A  is  transformed  to  the  Real  Jordan  Canonical  Form  [10]. 

We  presented  a  class  of  discrete-time  linear  systems  for  which  positively  in¬ 
variant  polyhedral  sets  are  described  by  the  Lyapunov  function  V{x)  =  \\Wx\\oo 
and  can  be  computed  very  efficiently.  However,  it  should  be  noted  that  in  our  sta¬ 
bility  analysis  for  switched  linear  systems,  it  is  not  necessary  for  the  individual 
invariant  polyhedral  sets  to  be  centrally  symmetric.  Positively  invariant  poly¬ 
hedral  sets  for  stable  discrete-time  systems  can  be  determined  using  computer 
generated  Lyapunov  functions  [6].  The  class  of  computer  generated  Lyapunov 
functions  has  been  used  for  stability  analysis  of  nonlinear  systems  in  [6,7,18,20]. 
The  main  idea  is  to  construct  a  Lyapunov  function  that  guarantees  the  stability 
of  a  set  of  matrices  that  is  determined  by  applying  Euler’s  discretization  method 
to  a  system  of  nonlinear  differential  equations. 

Our  approach  here  is  to  use  a  computer  generated  Lyapunov  function  for 
each  individual  subsystem.  Consider  the  matrix  A  €  and  let  Pq  C  3^^^  be 

a  bounded  polyhedral  region  of  the  origin.  We  denote  the  convex  hull  of  P  by 
conv(P).  Following  [6]  we  define 


and 


Pk  =  conv 


p*  =  U  Pi. 

t=0 


(9) 


(10) 


The  following  results  may  be  found  in  [6]:  First,  the  matrix  A  is  stable  if  and 
only  if  P*  is  bounded.  Second,  if  A  is  stable  then  each  set  Pk  can  be  computed 
by  Pk-i  using  finitely  many  iterations.  Furthermore,  it  is  shown  in  [7]  that  if 
there  exists  constant  A"  6  such  that  the  eigenvalues  of  A  satisfy  the  condition 
l^il  <  <  1,  then  the  set  P*  is  finitely  computable.  In  this  case  the  set  P* 

is  polyhedral  as  the  convex  hull  of  finitely  many  points.  Furthermore,  P*  is  a 
positively  invariant  set  of  the  system.  Then,  a  piecewise  linear  Lypunov  function 
can  be  defined  as  the  Lypunov  function  induced  by  the  set  P*. 


4  Stabilizing  Switching  Sequences 

In  this  section,  we  present  an  approach  based  on  multiple  Lyapunov  functions 
for  the  stability  analysis  of  the  switched  system  (1).  The  main  contribution  is 
an  efficient  characterization  of  a  class  of  switching  laws  of  the  form  (2)  which 
guarantee  the  stability  of  the  system.  We  assume  that  each  individual  subsystem 
admits  a  positively  invariant  polyhedral  set  that  contains  the  origin  which  is 
described  by 

Pg  =  {xen^:  W^x  <  1}  (11) 

where  G  and  1  =  [1, . . . ,  1]^  G  3?"^.  In  view  of  the  above  results,  such 

a  polyhedral  set  can  be  computed  if  the  there  exists  constant  K  such  that 
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the  eigenvalues  of  Aq  satisfy  the  condition  |Ai|  <  K  <  1.  We  denote  the  rows  of 

the  matrix  hy  i  =  l,...,mq.  The  Lyapunov  function  induced  by 

the  set  Pq  can  be  described  by 

=  ,  max  (12) 

We  consider  a  class  S  of  switching  sequences  that  can  be  described  by  s  = 
(90, <o),  (gi,*i),  •  •  • ,  •  •  • .  ^(^o)  =  ^0-  It  is  assumed  that  if  s  is  finite  then 

tj^i  =  00  and  that  qj  ^  Qj+i.  Such  a  sequence  can  be  generated  by  the  switching 
law  qj(tj  +  1)  =  S{qj-i{tj),x{tj)),  j  =  1,2, . , .. 

Proposition  1.  Consider  a  switching  sequence  s  G  S.  If  Vqjixitj  +  1)]  < 
Vq._^[x{tj)],  j  =  1,2,...,  then  the  switched  system  x{t  +  1)  =  Aqx{t)  is  sta¬ 
ble  in  the  sense  of  Lyapunov. 

Proof  Consider  the  multiple  Lyapunov  function  defined  by 

W))  =  n,W‘)l.  (13) 

then  by  the  definition  of  Vq.  we  have  that  for  every  t  >  to,  t  G  Z"*" 

DV{x)  =  V[x{t  +  1)1  -  V[x{t)]  <  0.  (14) 

Note  that  the  switched  system  for  a  fixed  switching  sequence  s  can  be  viewed  as 
a  time- varying  system.  Since  V (rr)  is  positive  definite  and  radially  unbounded, 
and  DV  negative  semidefinite,  the  system  is  stable  in  the  sense  of  Lyapunov;  see 
for  example  [1]. 

A  multiple  Lyapunov  function  composed  by  piecewise  linear  Lyapunov  func¬ 
tions  of  the  individual  subsystems  offers  a  significant  advantage.  It  allows  the 
characterization  of  the  switching  sequences  that  satisfy  the  condition  of  Propo¬ 
sition  1  by  computing  a  conic  partition  of  the  state  space. 

First,  we  briefly  describe  the  necessary  notions  and  notation  from  convex 
analysis  in  order  to  construct  the  conic  partition.  Given  a  polytope  P  €  5?",  then 
a  face  of  dimension  k  is  denoted  as  fc-face  F.  The  hyperplane  that  corresponds 
to  a  fc— face  F  is  defined  by  the  affine  hull  of  F  and  is  denoted  by  aff(P).  Each 
_  1)— face  corresponds  to  a  hyperplane  that  is  defined  by  aff(Pi)  =  {x  e 
:  {x,Wi)  =  1}  where  Wi  G  is  the  corresponding  gradient  vector.  The  set 
of  vertices  of  F  can  be  found  as  vert(P)  =  vert(P)  Pi  aff(P)  where  vert(P)  is 
the  set  of  vertices  of  the  polytope  P.  Finally,  we  denote  the  cone  generated  by 
the  vertices  of  F  by  cone(P).  Consider  a  pair  of  subsystems  with  matrices  Aq^ 
and  Aq^.  We  want  to  compute  the  region  =  {x  G  :  Vqzi^)  ^ 

Consider  the  faces  P?/  and  F^^  of  the  polytopes  Pq^  and  Pq^  respectively  and 
assume  that  C  ^  cone(P/^^)  H  cone{F^^)  0.  Next,  we  define  the  halfspace 
jpq2  =  {x  G  :  {x,  -wl\ )  <  0}  and  the  set  Q  =  CnH^^ .  It  is  shown  in  the 
following  lemma  that  the  multiple  Lyapunov  function  defined  in  Proposition  1 
is  decreasing  if  the  system  switches  from  qi  to  q2  while  x  £  f2. 

Lemma  1,  For  every  x  £  Q  we  have  that  Vq^ix)  <  Vq.^{x). 
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Proof.  For  every  x  £  C  the  Lyapunov  functions  for  the  subsystems  are  given 
by  Vg^{x)  =  {x,wfl)  and  Vq.^{x)  =  {x,wP)  respectively.  If  x  G  i?  we  have  that 
{x,wP  -w'fl)  <  0  since  x  G  and  therefore  Vg.^{x)  <  Vg^{x). 

Since  0  G  the  set  f?  is  a  clearly  a  polyhedral  cone  as  the  intersection 

of  cones  with  a  common  apex  {x  =  0)  as  shown  in  Fig.  2.  The  set  ^an  be 
computed  as  the  union  of  polyhedral  cones  by  repeating  the  above  procedure 
for  all  the  pairs  of  (n  —  1)— faces  of  the  polytope  P  as  shown  in  the 

following  algorithm. 


Fig.  2.  The  conic  partition  of  the  state  space. 

Algorithm  for  the  computation  of 
INPUT: 

for  ii  =  l,...,mq^ 
for  ^2  1}  •  •  ’  5 

C  ~  cone{Ff^ )  n  cone(F^^^ ); 
if  C  0  then 

=  {xg9?-:  {x,wr,~wll)<0} 

=  Q^gl  U  12; 

end 

end 

end 

The  above  procedure  can  be  repeated  for  every  pair  of  subsystems  to  iden¬ 
tify  a  class  of  stabilizing  switching  signals  for  the  switched  linear  system.  The 
class  of  switching  sequences  is  characterized  by  the  following  result.  Note  that  a 
numerical  example  that  illustrates  the  approach  may  be  found  in  [15]. 

Theorem  2.  Consider  the  class  of  switching  sequences  S  defined  by 

qj{tj  +  1)  =  5{qj„i{tj),x{tj)) 

^  0 


(15) 

(16) 
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for  j  =  1,2, _  The  switched  linear  system  x{t  +  1)  —  Agx{t)  is  stable  in  the 

sense  of  Lyapunov  for  every  switching  sequence  s  ^  S. 

Proof.  By  induction,  we  have  that  if  s  =  {^0,^0)  then  the  system  is  stable 
since  Ag^  is  stable.  Assume  that  the  switched  system  is  stable  for  the  switch¬ 
ing  sequence  s  =  {qo,to),{qi,ti),. . .  and  consider  the  sequence 

s'  =  Since  x{tj)  e  we  have  that 

Vg.[x(tj)]  <  Vg._.,[x{tj)].  Therefore,  the  multiple  Lyapunov  function  defined  by 
V[x{t)]  =  Vg.[x{t)],  tj  <  t  <  tj+i  is  decreasing  for  every  t  and  the  system  is 
stable  in  the  sense  of  Lyapunov. 

5  Continuous-Time  Switched  Linear  Systems 

In  this  section,  a  characterization  of  stabilizing  switching  sequences  for 
continuous- time  switched  linear  systems  is  presented.  The  set  of  stabilizing 
switching  sequences  is  characterized  by  computing  a  conic  partition  of  the  state 
space  similarly  to  the  discrete-time  case.  We  consider  the  switched  linear  system 

x(t)^Agx{t),  qeQ  (17) 

where  x(t)  6  and  Ag  G  The  switching  law  is  described  by 

=  S{q{t),x{t)).  (18) 

where  =  limr-^-t,  T>t  The  problem  is  to  identify  classes  of  switching  signals 
generated  by  (18)  for  which  the  system  (17)  is  stable.  Note  that  in  the  following  it 
is  assumed  that  only  finitely  many  switchings  can  occur  in  a  finite  time  interval. 


5.1  Background  Material 

In  order  to  study  the  stability  properties  of  the  switched  linear  system  (17), 
we  assume  that  each  individual  subsystem  admits  a  piecewise  linear  Lyapunov 
function  induced  by  a  positively  invariant  polyhedral  set.  Next,  we  summarize 
some  results  from  [13]  for  the  computation  of  piecewise  linear  Lyapunov  functions 
for  a  class  of  continuous-time  linear  systems.  Consider  the  continuous-time  linear 
system  x(t)  —  Ax{t)  where  x{t)  G  3?”'  and  A  G 

Similarly  to  the  discrete-time  case,  there  exists  a  class  of  continuous  lin¬ 
ear  systems  for  which  a  positively  invariant  polyhedral  set  can  be  computed 
very  efficiently.  If  the  eigenvalues  of  the  linear  system  satisfy  the  condition 
|Im{Ai}|  <  |Re{Ai}|  then  a  Lyapunov  function  V{x)  =  ||TTa:||oo  can  be  con¬ 
structed  using  a  similarity  transformation  [13]. 

The  use  of  piecewise  linear  Lyapunov  functions  for  the  stability  of  linear 
systems  is  based  on  the  following  result  [11].  Assume  that  there  exists  a  function 
V(x)  such  that  V  is  positive  definite  and  radially  unbounded,  and  the  upper 
right  Dini  derivative  [4]  of  V  satisfies  the  condition 

At^O  At 

Then,  the  equilibrium  j:  =  0  is  globally  asymptotically  stable. 
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The  conditions  for  V (x)  =  ||  VFxHoo  to  be  a  Lyapunov  function  for  the  system 
i:(t)  =  Ax(t)  can  be  stated  using  the  logarithmic  norm  induced  by  the  infinity 
norm.  The  logarithmic  norm  jUoo  of  a  matrix  Q  G  is  defined  as  [9] 


/ioo  =  lim 

Q^0+ 


l|/-aQ||oc-l 

a 


=  mp{9«+  XI  l%l}- 


(20) 

(21) 


Theorem  3.  fl3j  V(x)  =  is  a  Lyapunov  function  for  the  system  x  — 

Ax{t)  if  and  only  if  there  exists  Q  G  such  that  WA  -  QW  =  0  and 

a^oo(Q)  0. 


Corollary  2.  [13]  If  all  the  eigenvalues  \i  =  pLi±  (ji  of  the  order  system 
X  =  Ax{t)  satisfy  the  condition  \pLi\  <  \ai\,  then  there  exists  W  G  with 

rankW  =  n  such  that  the  polyhedral  set  P  =  {x  G  :  ||H^x||oo  <  1}  ^5  a 

positively  invariant  set  for  the  system. 

The  above  corollary  is  a  consequence  of  the  fact  that  the  matrix  equation 
^ A  —  QA  =  ^  has  a  solution  W  with  rankkL  =  n  if  and  only  if  the  eigenvalues 
of  A  are  identical  with  the  eigenvalues  of  Q  [10],  The  matrix  W  can  be  computed 
as  the  similarity  transformation  matrix  by  which  A  is  transformed  to  the  real 
Jordan  canonical  form  similar  to  the  discrete-time  case. 


5.2  Stabilizing  Switching  Sequences 

In  this  section,  we  present  an  approach  based  on  multiple  Lyapunov  functions 
for  the  stability  analysis  of  the  switched  system  (17).  We  assume  that  each  indi¬ 
vidual  subsystem  admits  a  piecewise  linear  Lyapunov  function  described  by  the 
infinity  norm.  The  main  contribution  is  an  efficient  characterization  of  a  class 
of  switching  laws  of  the  form  (2)  which  guarantee  the  stability  of  the  system. 
Similar  results  can  be  developed  for  more  general  piecewise  linear  Lyapunov 
functions  as  in  the  discrete-time  case  in  Section  4.  We  assume  that  each  indi¬ 
vidual  subsystem  admits  a  positively  invariant  polyhedral  set  that  contains  the 
origin  which  is  described  by 

P,-^{xG  :  \\W^x\\^  <  1}  (22) 

where  G  We  denote  the  rows  of  the  matrix  by  G  i  = 

l,...,n.  We  consider  a  class  S  of  switching  sequences  that  are  described  by 
5  =  (9o,to),  {qi,ti), . . . ,  {qj,tj), . . . ,  x{to)  =  xo  where  tj  G  =  0, 1, . . ..  It  is 
assumed  that  the  sequence  of  switching  instants  to,  ti,  •  •  • ,  tj, . . .  is  divergent  in 
the  sense  that  there  are  no  infinitely  many  switchings  in  a  finite  time  interval. 
Similarly  to  the  discrete-time  case,  it  is  assumed  that  qj  ^  A  sequence  s 
can  be  generated  by  the  switching  law  qj{tf)  =  5{qj^i{tj),x{tj)),  j  =  1,2, . . . . 
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Proposition  2.  Consider  a  switching  sequence  s  £  S.  If  Vq^l^itf)]  ^ 
Vq._,^[x{tj)],  j  =  1,2,...,  then  the  switched  system  x  —  Aqx(t)  is  stable  in  the 
567156  of  Lyapunov. 

Proof  Consider  the  multiple  Lyapunov  function  defined  by 

V[i(t)]  =  F,,[x{t)],  t,  (23) 

Then,  we  have 

V[x(t  +  At)]  -  V[x{t)]  ^ 

DV  =  lim  sup  <  0.  (24) 

^£^0  At 

for  every  t  £  and  therefore,  the  equilibrium  a:  =  0  is  stable  in  the  sense  of 
Lyapunov;  see  for  example  [11]. 

A  conic  partition  of  the  state  space  can  be  used  to  characterize  a  class  of 
switching  sequences  that  satisfy  the  condition  of  Proposition  2.  Consider  a  pair 
of  subsystems  with  matrices  and  Aq^ .  The  region  —  {x  G  :  Vg^ix)  < 
V^j(a;)}  can  be  computed  as  a  union  of  finitely  generated  cones  and  can  be 
computed  by  the  algorithm  presented  in  Section  4  similarly  to  the  discrete-time 
case.  The  class  of  stabilizing  switching  sequences  is  characterized  by  the  following 
result. 

Theorem  4.  Consider  the  class  of  switching  sequences  S  defined  by 

(25) 

^0  (26) 

/or  j  =  1,  2, . . ..  The  switched  linear  system  x  =  Aqx(t)  is  stable  in  the  sense  of 
Lyapunov  for  every  switching  sequence  s  £  S. 

Proof  Similar  to  the  proof  of  Theorem  2. 


Example  1.  Consider  the  switched  discrete-time  linear  system 

x  =  AqX{t),  q£  {1,2} 


(27) 


where 


1.7  1.8 

-4.5  -3.7 


and  A2  = 


0.7  -1 
1.6  -1.7 


(28) 


The  eigenvalues  of  the  matrices  Ai  and  A2  are  A  =  —  1  ih  .9/  amd  A  —  —.5  it  Aj. 
The  real  Jordan  canonical  form  can  be  computed  by  the  following  similarity 
transformations. 


Q^  =  W^Ai{WY^ 


-1  0.9 
-0.9  -1 


where  ~ 


2  1 
1  1 


(29) 


and 


Q2  =  W^A2{W^)-^ 


-0.5  0.4 
-0.4  -0.5 


where 


-1  1 
1  -0.5 


(30) 
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We  have  that  /ioo(Qi)  =  —0.1  <  0  and  therefore,  Vi{x)  =  ||W^x||oo  is  a 
Lyapunov  function  for  the  subsystem  Ai.  Similarly,  /Xoo((32)  =  -0.1  <  0  and 
V^{x)  = 

OO  is  a  Lyapunov  function  for  the  subsystem  A2.  The  functions 
Vi  and  V2  correspond  to  the  positively  invariant  polyhedral  sets 

Pi  =  {a:  e  <  1}  and  P2  =  {x  e  <  1}  (31) 

shown  in  Fig.  3(i). 


Fig.  3.  (i)  Positively  invariant  polyhedral  sets,  (ii)  The  region  O. 


Consider  the  faces  and  shown  in  Fig.  3(ii).  For  every  x  G  cone(F^)  n 
cone(p2)  we  have  that  Vi(x)  =  {x,w^)  and  ^2(1)  =  {x,w^)  with  u)i  =  [2, 1]  and 
=  [-1, 1]  respectively.  We  consider  the  halfspace 

Hf  =  {x  e  :  (x,  -  uji)  <  0}  (32) 

=  {r  6  5R2  . 

Therefore,  for  every  x  e  f?  =  cone(P‘)  n  cone(p2)  u  Hf  we  have  that  ^^(x)  < 
Vi{x). 

By  repeating  the  procedure  for  all  the  pairs  of  faces  for  the  polytopes  Pi  and 


P2  the  we  compute  the  region 

={xg5R^  y,,(x)<V„(x)}  (34) 

=  {x  e  3}^  :  xi  >  0}.  (35) 

Similarly  we  have  that 

r?|‘={xGK2:  v,.(x)<v;,,(x)}  (36) 

=  {x  G  :  xi  <  0}.  (37) 

Therefore,  for  any  switching  sequence  s  given  by  the  switching  law 

gj)*"'')  =  J(gi(t),x(t))  (38) 

^(t)  6  (39) 
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and 


(40) 

x(t)  e  (41) 

the  switched  system  is  stable.  A  stable  trajectory  is  shown  in  Fig.  4(i). 


Fig.  4.  (i)  A  stable  trajectory,  (ii)  An  unstable  trajectory. 


The  characterization  of  the  stabilizing  switching  sequences  is  based  on  suffi¬ 
cient  conditions.  Therefore,  for  a  switching  sequence  s  that  does  not  satisfy  the 
formulated  conditions,  the  switched  system  is  not  necessarily  unstable.  How¬ 
ever,  the  switched  system  (27)  can  generate  unstable  trajectories  as  shown  in 
Fig.  4(ii).  An  unstable  trajectory  can  be  generated  by  requiring  that  the  system 
will  keep  switching  indefinitely  and  that  the  Lyapunov  function  is  increasing  at 
every  switching. 

6  Conclusions 

In  this  paper,  a  class  of  stabilizing  switching  sequences  for  switched  linear  sys¬ 
tems  is  characterized  by  computing  conic  partitions  of  the  state  space.  The  main 
advantage  of  the  approach  is  that  the  methodology  for  computing  switching  laws 
that  guarantee  stability  is  based  on  the  parameters  of  the  system  and  so,  trajec¬ 
tories  for  particular  initial  conditions  do  not  need  to  be  calculated.  Therefore, 
the  proposed  approach  can  be  used  very  efficiently  to  investigate  the  stability 
properties  of  practical  hybrid  systems. 
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Abstract.  Our  study  is  concerned  with  a  particular  class  of  hybrid 
dynamical  systems,  namely  systems  with  discontinuous  vector  fields. 
We  will  show  that  such  systems  can  exhibit  a  novel  class  of  bifurcations 
which  are  not  observed  in  smooth  dynamical  systems.  Particularly, 
we  concentrate  on  bifurcations  which  arise  due  to  the  existence  of 
so-called  sliding  motion.  Using  appropriate  discrete  mappings  we  show 
the  possible  existence  of  complex  transitions  which  we  term  sliding, 
multisliding  and  grazing-sliding  bifurcations.  Relay  feedback  systems 
are  used  as  a  representative  example. 

Keywords:  Hybrid  Systems,  Bifurcations,  Sliding  motion 


1  Overview 

Hybrid  control  strategies  are  increasingly  used  in  applications.  The  resulting 
dynamical  systems  are  characterised  by  a  combination  of  continuous  and  discrete 
dynamics  which  can  give  rise  to  a  unique  class  of  phase  space  transitions.  A 
particularly  interesting  class  of  hybrid  systems  of  relevance  in  applications  is 
that  of  switched  dynamical  systems.  Examples  include  systems  with  dry  friction 
[1][2],  systems  with  impacts  (impact  oscillators,  vibroimpact  systems)  [3]  and 
relay  feedback  systems  [4]. 

Under  certain  conditions  these  systems  can  exhibit  solutions  lying  within 
their  discontinuity  set  or  sliding.  Numerical  and  experimental  evidence  of  dy¬ 
namical  transitions  involving  sliding  was  recently  reported  in  the  literature.  Ex¬ 
amples  include  the  formation  of  “chattering  orbits”  in  parallel  resonant  power 
electronics  converters  [5],  the  onset  of  stick-slip  motion  in  friction  oscillators 
[6]  and  the  the  occurence  of  fast  switching  periodic  solutions  in  relay  feedback 
systems  [4].  These  transitions  can  be  consistently  classified  in  terms  of  the  bi¬ 
furcation  scenarios  introduced  in  this  paper.  Their  occurrence  can  be  explained 
in  terms  of  the  interaction  between  the  system  i?-limit  set  and  the  phase-space 
manifold  where  sliding  is  possible,. 

In  what  follows,  we  will  focus  our  attention  on  linear  systems  with  a  relay 
feedback  element.  Although,  systems  with  a  relay  feedback  have  been  studied  for 
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a  long  time  [7],  [8]  the  dynamics  of  these  systems  is  not  fully  understood.  Even 
low-order  relay  feedback  systems  can  exhibit  complex  self  oscillations,  which 
include  periodic  solutions  with  segments  of  sliding  motion  [9j,  [10],  [11].  The  aim 
of  this  paper  is  to  use  relay  feedback  systems  as  a  representative  example  to 
describe  a  novel  class  of  bifurcations  involving  sliding  which  can  be  observed  in 
a  wider  class  of  switched  and  hybrid  dynamical  systems  [12]. 

The  outline  of  the  paper  is  the  following.  The  general  form  of  systems  with 
a  relay  feedback  under  investigation  is  introduced  in  section  2  and  appropriate 
maps  used  for  numerical  investigation  are  defined.  Section  3  illustrates  the  four 
possible  cases  of  novel  bifurcations  caused  by  the  interaction  of  the  system  f2- 
limit  set  with  the  region  where  sliding  is  possible.  Then,  in  section  4  the  detailed 
analysis  of  a  third-order  representative  example  is  presented.  Evidence  of  chaotic 
attractors  are  given  and  appropriate  one-dimensional  mappings  are  derived  to 
study  their  nature.  Finally,  in  section  5  after  drawing  some  conclusions,  we  give 
suggestions  for  further  work. 

2  Background 


In  what  follows,  we  consider  a  class  of  systems  with  discontinuous  vector  field 
corresponding  to  single-input,  single-output,  linear  time-invariant  (LTI)  systems 
with  unit  negative  feedback  of  the  output  variable.  The  systems  under  investi¬ 
gation  have  the  following  general  form: 


X  =  Ax  +  Bu, 

(1) 

y  =  Cx, 

(2) 

u  =  -sgn{y), 

(3) 

where  A  G  €  i2”^^^and  C  €  constant  matrices.  The  input  u 

and  output  y  of  the  linear  part  are  scalar  functions,  while  x,  the  state  vector, 
has  n  >  1  components.  The  “  sgn”  function  (which  is  the  non-linear  term  in  the 
system  equations)  is  defined  as  sgn{y)  -  1,  if  ^  >  0,  sgn(2/)  =  -1,  if  t/  <  0  and 
sgn(?/)  G  (-1,1),  if  ^  =  0. 

It  is  assumed  that  the  system  matrices  are  given  in  observer  canonical  form,  i.e.: 


A  = 

/  -ai  1  0  •••  0^ 
-02  0  1  •  •  •  0 

,  B  = 

fh\ 

b2 

,  c  = 

fl\ 

0 

,  -o„_i  0  0  0  1 
^  -a„  0  0  0  0/ 

64 

\h) 

0 

\0j 

The  above  matrices  correspond  to  the  following  transfer  function: 

G(s)  =  C{sl  -  A)-^B  = _ +  ■  ■  ■  +  bn~is  +  b„ 

s’'  +  +  023"-^  H - h  On-is  +  a„ ' 


(5) 


On  a  Novel  Class  of  Bifurcations  in  Hybrid  Dynamical  Systems  363 


The  system  trajectory  generated  by  the  vector  field  characterized  by  equations 
{l)-(3)  is  smooth  and  continuous  in  the  two  subspaces,  Hi  and  defined  as: 

Hi^{x€R^:Cx>  0},  (6) 

H2  =  {xe  :  Cx  <  0}.  (7) 

System  (l)-(3)  switches  from  one  (LTI)  region  {Hi  or  H2)  to  the  other  whenever 
the  system  trajectory  crosses  the  switching  hyperplane  S  defined  as: 


S  =  {x€R^:Cx  =  0}.  (8) 

For  all  initial  conditions  outside  5,  the  system  trajectory  will  ultimately  cross 
S  assuming  positive  and  stable  steady-state  gain  G(0)  [9].  Note  that  the  system 
under  investigation  is  symmetric  with  respect  to  the  origin. 


2.1  Sliding  Motion 

Systems  such  as  (l)-{3)  can  exhibit  a  very  peculiar  type  of  motion  termed  slid¬ 
ing.  This  corresponds  to  a  solution  lying  within  the  system  discontinuity  set 
S.  Heuristically,  sliding  can  be  seen  as  characterised  by  an  infinite  number  of 
switchings  between  the  two  subspaces  Hi  and  H2.  Sliding  motion  is  only  possible 
when  the  vector  field  points  towards  the  switching  manifold  S  in  both  regions 
Hi  and  H2  (see  fig.  1).  Thus  by  studying  the  direction  of  the  vector  field  in  a 
neighborhood  of  the  switching  manifold,  it  is  possible  to  identify  a  set  C  S 
where  sliding  is  possible.  We  term  S3  as  the  sliding  region.  Any  trajectory  hitting 
the  switching  manifold  in  S3  is  constrained  to  evolve  on  it  until  the  trajectory 
reaches  the  point  where  the  vector  field  changes  its  direction  on  the  boundary  of 
the  sliding  region  (see  figure  1).  Using  the  equivalent  control  method  presented 
in  [13],  we  can  obtain  the  dynamical  system  describing  the  motion  of  a  trajectory 
within  the  region  S3.  The  equivalent  control  input  Ueq  G  (— 1, 1)  is  defined  as  the 


Fig.  1.  Schematic  representation  of  the  phase  space  topology  in  the  case  n  =  3 


364  P.  Kowalczyk  and  M.  di  Bernardo 


controller  that  keeps  the  trajectory  on  the  switching  hyperplane,  i.e.  the  control 
input  that  guarantees  y  =  0  and  7/  =  0.  Using  (l)-(3),  it  can  be  shown  that  for 
the  system  under  investigation  such  control  input  is  given  by: 

Ueq  —  -{CB)~^CAx.  (9) 

By  substituting  (9)  into  (1)  we  obtain  the  set  of  equations  describing  the  system 
dynamics  within  the  sliding  region,  which  is  given  by: 

X  =  Ax,  (10) 

where  A  =  [I  —  {CB)~^BC]A  and  I  denotes  the  n  x  n  identity  matrix. 

According  to  the  direction  of  the  vector  field,  we  can  define  regions  on  the 
hypersurface  S  where  7/  =  0,  7/  >  0  and  y  <  0  respectively.  Namely,  we  define 
Si  =  {xe  :  CAx  >  CB},  S2  =  {x  G  :  CAx  <  -CB},  83  {x  e  R^  : 
\CAx\  <  CB}. 

Additionally,  we  define  the  boundary  between  S3  and  Si  which  we  denote 
as  ^531,  and  the  boundary  between  S3  and  S2  which  we  denote  as  8832,  as: 
8831  ==  {x  €  R^  :  CAx  =  CB},  8S32  =-  {x  e  R^  :  CAx  =  -CB}.  Note  that 
sliding  is  only  possible  when  the  sliding  set  ^3  is  non-empty  i.e.  when  CB  >  0, 
in  (13). 

2.2  Self  Oscillations  and  Poincare  Maps 

Typically,  a  system  with  relay  feedback  has  self-oscillations  [14].  This  corre¬ 
sponds  to  the  periodic  switching  of  the  system  trajectory  between  Hi  and  H2 
(an  example  for  a  third-order  system  is  presented  in  figure  2).  The  dark  region  in 
the  figure  indicates  the  region  on  the  switching  manifold  S  where  the  trajectory 
of  the  system  slides  (the  region  denoted  as  ^3  in  figure  1). 

As  shown,  in  [10]  using  the  system  explicit  solutions  we  can  characterise  the 
system  evolution  by  using  an  appropriate  set  of  discrete-time  maps.  Namely  we 
can  define,  the  upper  switching  map  77+  :  5  as  the  mapping  which  de¬ 

scribes  the  dynamics  from  xq  E  Si  to  xi  e  S  (see  figure  3).  We  also  define  the 
lower  switching  map  77“  :  S2  S2  ^  S  as  the  map  which  describes  the  system 
dynamics  from  a  point  xi  E  S2  to  X2  E  S.  Finally,  we  define  the  overall  switching 
map  77  as  the  composition  of  77+  and  77“ :  77  =  77+  o  77“ .  The  map  77  can  be 
used  to  analyze  simple  periodic  orbits. 

As  discussed  in  [10]  sliding  sections  can  become  part  of  an  orbit.  To  in¬ 
vestigate  the  behaviour  of  orbits  with  a  section  lying  in  the  sliding  surface 
we  introduce  a  map  B  which  maps  the  points  in  region  S3  to  its  bound¬ 
ary  8S3  :=  8S31U8S32.  Once  the  trajectory  reaches  the  boundary  of  the 
sliding  section,  it  leaves  the  switching  region.  We  define  B  as  the  mapping 
which  takes  a  point,  x,  from  region  S3  to  a  point  xi  on  its  boundary,  i.e. 
B  :  R^  — >  72^  :  53  f->  883.  The  simplest  symmetric  orbit  with  sliding  is  de¬ 
picted  in  figure  3.  Note  the  existence  of  two  sliding  segments  per  period  (the 
orbit  is  symmetric).  Stable  asymmetric  orbits  are  also  possible  [16]. 
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Fig.  2.  Typical  trajectory  (corresponding  to  the  self-oscillations)  of  the  system  with 
relay  feedback  (l)-(3) 


Fig.  3.  The  simple  symmetric  orbit  with  two  sliding  segments 


The  orbit  depicted  in  figure  3  can  be  described  by  the  composition  of  mappings 
77+ ,  n~  and  E.  Orbits  with  higher  number  of  sliding  sections  and  asymmetric 
orbits  can  be  defined  by  an  appropriate  composition  of  the  mappings  77+ ,  77“ 
and  E  (see  [10]  for  further  details). 

3  Bifurcation  Scenarios  Involving  Sliding  Section 

We  now  introduce  the  possible  bifurcation  scenarios  involving  the  interaction 
between  trajectories  of  the  system  and  the  sliding  region  53.  These  scenarios  were 
identified  after  careful  numerical  and  analytical  investigation  and  were  partly 
reported  independently  for  the  first  time  in  [10]  and  [15]. 
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We  distinguish  four  possible  cases  of  such  bifurcations  involving  sliding,  which 
we  will  term  generically  as  sliding  bifurcations  (see  figure  4),  Without  loss  of 
generality,  we  assume  that  such  local  bifurcations  involve  sections  of  trajectories 
belonging  to  some  periodic  orbit  of  the  system.  Figure  4(a)  depicts  the  so-called 


Fig.  4.  The  four  possible  bifurcation  scenarios  involving  collision  of  a  segment  of  the 
trajectory  with  the  boundary  of  the  sliding  region  dSs 


sliding  bifurcation  of  type  A.  This  corresponds  to  the  following  scenario. 
When  a  control  parameter  is  varied  the  trajectory  hits  transversally  the  bound¬ 
ary  of  the  sliding  strip  dSs  (trajectory  2  in  figure  4(a)).  Further  variation  of 
the  parameter,  make  the  trajectory  enter  the  sliding  region  ^3,  thus  causing 
the  onset  of  sliding  motion.  Note,  that  the  trajectory  leaves  the  sliding  strip 
tangentially  (i.e  y  =  0). 

In  the  case  presented  in  figure  4(b),  instead,  the  trajectory  grazes  tangentially 
the  boundary  of  the  sliding  strip,  dSs,  from  the  subspace  Hi  (or  H2).  Again, 
this  causes  the  formation  of  a  section  of  sliding  motion.  We  term  this  transition 
as  a  grazing-sliding  bifurcation. 

The  third  scenario  depicted  in  figure  4(c)  is  somehow  similar  to  a  sliding  bi¬ 
furcation  (case  4(a)).  In  case  4(a),  though,  the  trajectory  zooms  off  the  switching 
manifold  S  at  the  bifurcation  point  while  in  case  4(c)  it  stays  within  the  slid¬ 
ing  region.  Specifically,  these  two  bifurcation  events  differ  by  the  sign  of  y  at 
the  boundary  of  the  sliding  strip  (^53).  We  call  case  4(c),  a  switching-sliding 
bifurcation,  or  sliding  bifurcation  type  B. 
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The  last  case  is  termed  a  multisliding  bifurcation  and  is  depicted  in  fig¬ 
ure  4(d).  It  differs  from  the  scenarios  presented  above  since  the  segment  of  the 
trajectory  which  undergoes  the  bifurcation  lies  entirely  in  the  sliding  region  S3. 
Namely,  through  the  variation  of  some  parameters,  a  sliding  segment  hits  tan¬ 
gentially  the  boundary  of  the  sliding  strip.  Further  variations  of  the  parameter, 
cause  such  sliding  segment  to  “fall”  off  the  sliding  region  causing  the  formation 
of  an  additional  segment  of  the  trajectory  lying  in  the  Hi  or  H2  subspace. 

We  now  give  analytical  conditions  for  each  of  these  bifurcations  in  terms  of 
the  properties  of  the  system  output,  y,  at  the  bifurcation  point,  say,  x.  Similar 
conditions  were  also  reported  independently  in  the  Russian  Literature  in  [15]. 


3.1  Analytical  Conditions  for  Sliding  Bifurcations 

For  each  case  of  the  four  scenarios  reported  above,  the  following  conditions  must 
hold  at  the  bifurcation  point,  say  x: 

1.  Cx  =  0, 

2.  CAx  -  CB. 

Condition  1  ensures  that  at  the  bifurcation  point  the  trajectory  lies  on  the 
switching  manifold.  Condition  2  corresponds  to  the  fact  that  the  bifurcation 
point  belongs  to  the  boundary  of  the  sliding  strip.  Note,  that  condition  2  also 
implies  that  y  =  CAx  -  CB  =  0  i.e.  the  trajectory  must  leave  S3  tangentially. 
Additional  conditions  involiving  higher  order  derivatives  of  y  can  be  given  for 
each  of  the  scenarios.  Namely  we  have  the  following  extra  conditions. 

Sliding  bifurcation  type  A.  (figure  4(a)) 

In  addition  to  conditions  1  and  2  we  note  that  in  this  case  the  trajectory  moves 
toward  the  boundary  of  S3  at  the  bifurcation  point  x.  Thus,  we  have  i/  >  0,  i.e. 

CA^x  -  CAB  >  0.  (11) 


Grazing  while  sliding  bifurcation,  (figure  4(b)) 

As  in  the  previous  case,  the  trajectory  moves  away  from  the  sliding  region  ^3 
at  a  grazing-sliding  bifurcation  point,  x.  Hence,  in  addition  to  the  two  general 
conditions  1  and  2,  we  also  require  condition  (11)  to  hold. 

Multisliding  bifurcation,  (figure  4(c)) 

In  this  case  the  bifurcating  trajectory  hits  the  boundary  of  the  sliding  strip 
tangentially.  Thus,  at  x  we  must  have  y  =  0,y  >  0,  i.e. 


CA'^x  -  CAB  =  0 

CA^x  -  CA‘^B  >  0 


(12) 

(13) 
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Switching  sliding,  sliding  type  B  bifurcation,  (figure  4(d)) 

Finally,  in  this  case,  the  trajectory  moves  towards  the  interior  of  S3  (away  from 
dSsi)  at  the  bifurcation  point,  x,  thus  ^  <  0  and  we  have 

CA^x  ~  CAB  <  0.  (14) 

Similar  conditions  can  be  given  if  intersections  with  0832  are  considered.  A 
generalization  of  these  conditions  to  the  case  of  n-dimensional  PWS  systems  of 
the  form: 


X  = 


Fi{x)  if  H(x)>0, 
F2{x)  if  H{x)<0 


where  x  €  ^1,^2  •  )— >  are  sufficiently  smooth  in  the  region  ofinterest 

and  H  :  R  is  a  scalar  function  of  the  system  states,  can  be  found  in  [12]. 


4  Numerical  Analysis  of  a  Third  Order  Representative 
Example 

The  state  space  representation  of  the  third-order  relay  feedback  system,  which 
will  serve  as  a  representative  example  is  characterised  by  the  matrices: 

/  -{2Cw  +  X)  10\  f  k  \  /1\^ 

71=  -(2Ca.A  +  a<2)0  1  ,  B  =  \  2kpa  \  C  =  0  .  (15) 

\  -Acc;2  0  0/  \kp^  )  \oJ 

The  above  state-space  representation  corresponds  to  the  following  transfer  func- 
tion: 


G{s)  =  k 


-1-  2a ps  -h 

(s2-f  2Ca;s  +  a;2)(5-f  A)' 


(16) 


The  parameters  uj  and  ^  denote  the  natural  frequency  and  the  damping  of  the 
complex  pair  of  poles  while  p  and  a  represent  the  corresponding  quantities  for 
the  complex  pair  of  zeros,  -A  is  the  location  of  the  real  pole  and  ^  is  the 
steady- state  gain. 


4.1  Sliding  Bifurcation 

As  mentioned  in  the  previous  section,  we  can  observe  the  transition  from  a 
generic  orbit  to  an  orbit  with  sliding  by  varying  the  system  parameters.  We 
present  examples  of  these  “sliding  bifurcations”  for  the  third  order  relay  system 
(15)  according  to  the  four  distinct  scenarios  introduced  above. 

Figure  5(a)  represents  a  stable  symmetric  orbit  (before  its  transition  to  an 
orbit  with  sliding)  for  the  following  values  of  the  parameters,  C  =  u^  =  A  =  -(7  = 
k  =  I,  Here  p  is  varied  in  a  neighborhood  of  po  =  3. 
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(a) 


(b) 


Fig.  5.  The  scenario  of  sliding  bifurcations  of  the  simple  orbit  for  the  parameter  values: 
—a  =  \  =  k  —  ~  l,pisa  varied  parameter  with  subsequent  values  p  =  3  fig. (a), 

p  =  2.1  fig.(b)  and  p  =  1  for  fig(c),  (d)  projection  of  the  trajectory  on  the  xi,X2  plane 
showing  that  the  orbit  does  intersect  the  boundary  of  the  sliding  strip  at  the  bifurcation 
point 


As  the  parameter  p  is  decreased,  this  simple  orbit  hits  the  boundary  of  the 
sliding  segment  ^^3  transversally  (see  figures  5-(a)  and  (b)).  Through  this  slid¬ 
ing  bifurcation  of  type  A,  variation  of  the  parameter  p  ,  cause  the  formation  of 
a  sliding  orbit.  To  investigate  the  stability  of  the  orbits  which  undergo  sliding 
bifurcation  the  eigenvalues  of  a  the  point  mappings  77+  and  E  o  77+  were  com¬ 
puted.  The  orbits  presented  in  figures  4(a)  -  4(d)  are  symmetric  orbits.  Thus, 
it  suffices  to  compute  the  eigenvalues  of  the  fixed  points  associated  to  these 
solutions  using  either  the  lower  or  the  upper  switching  map,  77“  or  77+  (appro¬ 
priately  composed  with  E  when  sliding  orbits  are  considered).  Figs.  6(a)  and 
6(b)  show  the  two  significant  eigenvalues  of  a  fixed  point  (corresponding  to  a 
symmetric  orbit). 

The  region  denoted  as  2  in  figures  6(a),  6(b)  corresponds  to  the  fixed  point 
associated  with  a  simple  symmetric  orbit.  The  region  labeled  as  1  in  figures  6(a), 
6(b)  corresponds  to  the  fixed  point  associated  with  a  sliding  orbit.  Note  the 
apparent  piecewise  smoothness  in  the  value  of  eigenvalues  of  the  fixed  points  of 
orbits  before  and  after  the  sliding  bifurcation.  One  eigenvalue  becomes  identically 
zero  at  the  sliding  bifurcation  point  (fig.  6(b)).  Hence,  there  is  only  one  significant 
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Fig.  6.  absolute  values  of  eigenvalues  of  a  map  /7+  and  o  Z*  before  (part  of  graph 
6(a),  6(b)  denoted  as  2  and  after  sliding  bifurcations  part  of  the  graph  6(a),  6(b) 
denoted  by  1 


eigenvalue  characterising  sliding  orbits  in  these  three-dimensional  system.  These 
orbits  can  be  analyzed  using  the  map  Q  :  dS^,  >— >  ^5^3  defined  above  which  in 
this  case  is  indeed  one-dimensional, 

4.2  Multisliding  Bifurcation 

In  this  subsection,  we  consider  the  bifurcation  scenario  which  we  termed  mul¬ 
tisliding  (see  figure  4(a)).  Let  us  denote,  by  A  an  orbit  before  the  multisliding 
bifurcation  and  B  an  orbit  after  its  occurience  (thus  both  orbits  differ  by  the 
number  of  sliding  segments).  We  also  assume  that  upper  case  denotes  stable 
orbits  and  lower  case  unstable  ones.  Close  to  a  multisliding  bifurcation  point 
(fig.  4(d))  we  can  observe  two  type  of  transitions  A  B  or  A,b  {^}.  The 
transition  A  ^  B  corresponds  to  a  transcritical-like  bifurcation  scenario  where 
a  new  sliding  orbit  with  a  different  number  of  sliding  segments  is  born  at  the 
bifurcation  point.  The  other  case  A,b  corresponds  to  the  case  when  two 

orbits,  one  stable  and  the  other  unstable,  collide  and  disappear  on  the  boundary. 
The  transcritical-like  transition,  ^  of  a  multisliding  orbit  is  shown  in  figure 
7.  The  parameters  have  the  following  values:  C  =  0.05,  p  =  -a  =  A;  =  A  =  l. 
The  parameter  uj  is  varied  and  takes  the  value  10.14  in  fig.  7(a),  10.24  in  fig. 
7(b)  and  10.74  in  figure  7(c).  It  was  mentioned  in  section  2  that  the  orbits  with 
sliding  section(s)  can  be  analyzed  using  appropriate  one-dimensional  mappings 
from  the  line  dS^  back  to  itself.  Fig.  7(d)  shows  the  one-dimensional  map  ob¬ 
tained  by  varying  the  xz  coordinate  on  the  line  3:2  =  1  (xi  =  0)  and  applying 
the  maps  77+,  Z  and  n~ .  The  proper  composition  of  these  maps  drives  the 
point  from  the  line  X2  =  l{xi  =  0)  back  to  itself.  Note,  the  existence  of  a  kink 
in  figure  7(d).  This  is  the  effect  of  the  multisliding  bifurcation.  The  multisliding 
transition  of  type  0  A,  b  and  the  corresponding  one-dimensional  map  from 
the  line  dS^  back  to  itself  is  shown  in  fig.  8.  Here,  the  orbit  was  obtained  for  the 
following  parameter  values: -a  =  p  =  A:  =  1,A  =  0.05,  cj  =  10;  while  C  is  varied 
in  a  neighborhood  of  Co  =  0.0395. 
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Fig.  7.  Figures  7(a)-7(c)  represent  the  scenario  of  multisliding  bifurcation.  Note  the 
tangency  of  the  orbit  7(b)  to  the  boundary  of  the  sliding  segment  dSz.  Figure  7(d)  rep¬ 
resents  the  map  from  the  line  dSsi  back  to  itself  for  the  parameter  values  corresponding 
to  the  orbit  depicted  in  figure  7(b) 


Fig.  8.  Symmetric  orbit  with  multiple  amount  of  sliding  sections  (a)  and  corresponding 
1-dimensional  map  (b) 


4.3  Grazing  while  Sliding  Bifurcation 

We  now  present  numerical  evidence  for  the  so-called  grazing-sliding  bifurcation  - 
figure  4(b).  A  trajectory  undergoes  the  grazing-sliding  transition  when  a  segment 
of  the  trajectory  touches  tangentially  the  boundary  dS^  of  the  sliding  segment 
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Fig.  9.  Symmetric  orbit  with  multiple  amount  of  sliding  sections  before  (a)  and  af¬ 
ter  (b)  grazing  while  sliding  bifurcations,  (c)(d)  close  up  of  the  region  where  part  of 
the  trajectory  graze  the  boundary  of  the  sliding  strip  -  accordingly  before  and  after 
bifurcation 


from  the  subspace  Hi  or  H2.  After  this  bifurcation,  the  trajectory  contains  an 
additional  sliding  segment  (figures  4(a)-4(c)).  In  the  case  presented  here,  as  the 
parameter  C  is  increased,  one  of  the  loops  making  up  the  orbit  (figure  9(a)) 
changes  its  shape.  This  in  turn  causes  (with  further  variation  of  the  control 
parameter  C)  the  loop  to  touch  the  boundary  of  the  sliding  strip  from  above 
(and  below  -  note  the  symmetry  of  the  transition  scenario)  and  enter  the  sliding 
strip,  (figure  9(b)).  The  parameter  values  for  which  the  transition  described  was 
detected  take  the  following  values:A  =  0.05,  k  =  —a  =  p  =  =  10.  C  is  varied 

between  0.025  and  0.032. 

4.4  Switching  Sliding  Bifurcation,  Sliding  Type  B 

Despite  several  attempts  switching-sliding  bifurcations  of  stable  periodic  solu¬ 
tions  were  not  detected  for  the  third-order  system  under  investigation.  Evidence 
of  their  occurence  in  a  second-order  friction  oscillator  can  be  found  in  [15]. 


4.5  Chaos 

The  seemingly  simple  system  which  serves  us  as  an  example  is  also  found  to 
exhibit  chaotic  behaviour.  The  chaotic  attractor  depicted  in  fig.  10  is  obtained 
for  the  following  parameter  values:  C  =  -0.08,  uj  =  10,  k  =  p  =  -a  =  1  and 
A  =  0.05.  Applying  the  idea  of  1-dimensional  point  mappings  from  the  line 
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Fig.  10.  Chaotic  attractor  and  corresponding  1-dimensional  map 


dS^i  back  to  itself  one  can  study  this  chaotic  evolution  by  considering  the  one¬ 
dimensional  map  which  presented  in  figure  10(b),  Note  the  characteristic  shape 
of  the  kinks  of  the  map  due  to  near-multisliding  events  in  the  trajectory. 

The  map  shown  in  figure  10(b)  has  interesting  dynamics,  similar  to  those 
of  the  double  iteration  of  the  tent  map  [17],  Thus,  the  occurence  of  chaos  in 
the  system  can  be  explained  as  resulting  from  the  merging  of  two  asymmetric 
chaotic  attractors.  It  is  relevant  to  point  out  that  the  formation  of  the  attractor 
is  organized  by  the  occurence  of  the  sliding  bifurcations  presented  in  the  paper 
(see  [16]). 

5  Conclusions  and  Future  Work 

It  has  been  shown  by  means  of  a  representative  example  that  very  complex 
dynamics  can  be  observed  in  systems  with  discontinuous  vector  field.  Evidence 
of  novel  bifurcations  was  given  ,  namely  sliding,  multisliding  and  grazing-sliding 
bifurcations.  Our  numerical  analysis  details  their  occurrence  in  a  third-order 
relay  feedback  system.  We  show  that  these  novel  transitions  lead  to  the  formation 
of  the  chaotic  attractor  presented  in  fig.  10. 

All  these  novel  bifurcations  can  be  studied  analytically  by  means  of  the 
Poincare  maps  we  introduced.  Current  work  to  be  presented  elsewhere  [12]  is 
aimed  at  carrying  out  the  analytical  investigation  of  these  transitions  for  a  gen¬ 
eral  class  of  n-dimensional  PWS  systems  while  deriving  appropriate  normal  form 
maps  in  a  neighborhood  of  the  bifurcation  point.  This  will  allow  a  classification 
of  all  possible  bifurcation  scenarios  following  one  of  the  transitions  presented  in 
this  paper. 

We  conjecture  that  the  bifurcations  described  in  this  paper  are  common  in 
applications  involving  a  wider  class  of  switched  dynamical  systems  with  sliding. 
Moreover,  we  anticipate  that  they  are  an  important  mechanism  leading  to  the 
formation  of  deterministic  chaos  and  other  complex  behaviour  in  hybrid  dynam¬ 
ical  systems. 
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Abstract.  In  this  paper  we  investigate  the  question  of  the  global  con¬ 
trollability  posed  for  control  hybrid  systems  with  autounomous  and  con¬ 
trolled  swithchings.  The  main  tool  for  our  analysis  is  the  notion  of  the 
controlled  hyhrifold.  New  sufficient  conditions  for  the  global  controllabil¬ 
ity  are  obtained  in  terms  of  the  so-called  hybrid  fountains. 


1  Introduction 

In  this  paper  we  consider  systems  which  have  a  hybrid  nature,  in  the  sense  that 
the  dynamics  of  the  system  combines  continuous  and  discrete  components.  We 
model  control  hybrid  systems  as  a  tuple  consisting  of  a  state  space,  a  set  of 
admissible  continuous  and  discrete  controls,  a  family  of  controlled  vector  fields 
assigned  to  each  discrete  state,  a  collection  of  autonomous  and  controlled  switch¬ 
ing  surfaces,  and  a  collection  of  the  correspondint  reset  maps. 

The  main  question  investigated  in  the  paper  is  the  controllability  of  control 
hybrid  systems.  This  issue  has  been  addressed  in  [1,5,12,13].  In  particular,  in 
[12],  the  notion  of  controllability  for  hybrid  systems  is  formalized  by  continuity 
of  system  functions.  In  [1],  the  authors  derive  a  necessary  and  sufficient  algebraic 
condition  for  a  certain  subclass  of  piecewise  affine  hybrid  systems.  In  [13],  a 
sufficient  condition  for  controllability  of  hybrid  systems  is  formulated  in  terms 
of  the  so-called  arrival  sets. 

Because  of  the  complexity  of  the  problem  of  the  global  controllability,  its 
unlikely  to  find  uniform  sufficient  conditions  for  general  hybrid  systems.  Thus, 
we  restrict  our  study  to  a  special  subclass  of  control  hybrid  systems,  namely, 
the  systems  that  can  be  represented  as  hybrifolds.  The  notion  of  the  hybrifold 
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was  originally  introduced  in  [14]  and  extended  to  control  hybrid  systems  with 
autonomous  switchings  in  [9]  (see  also  [6],  where  the  hybrifold  notion  is  used  in 
problems  of  optimal  control  for  hybrid  systems).  In  this  paper  we  generalize  the 
results  formulated  in  [9]  to  systems  that  admit  both  autonomous  and  controlled 
switchings.  New  sufficient  conditions  for  the  global  controllability  are  obtained  in 
terms  of  the  so  called  hybrid  fountains.  The  advantage  of  the  approach  proposed 
in  this  paper  is  in  the  fact,  that  the  fountain  property  can  be  verified  at  each 
particular  state  and,  hence,  there  is  no  need  to  invoke  a  dynamic  programming¬ 
like  procedure  to  determine  arrival  sets  of  the  system. 

The  paper  is  organized  as  follows.  In  Section  2,  we  formally  define  the  class 
of  control  hybrid  systems  H  under  our  consideration  and  specify  the  standard 
assumptions  on  the  continuous  and  discrete  parts  of  the  dynamics  of  H.  In  Sec¬ 
tion  3,  we  generalize  the  notion  of  the  hybrifold  to  control  hybrid  systems  with 
controlled  and  autonomous  switchings  and  define  a  controlled  flow  on  the  hybri¬ 
fold.  Section  4  relates  the  global  controllability  of  H  to  the  global  controllability 
of  the  associated  controlled  hybrifold.  In  Section  5,  we  introduce  the  notion  of 
a  hybrid  fountain  and  provide  new  sufficient  conditions  for  the  global  controlla¬ 
bility  of  control  hybrid  systems. 


2  Regular  Control  Hybrid  Systems:  Standing 
Assumptions 

We  consider  control  hybrid  systems  which  in  this  paper  are  taken  to  be  of  the 
following  form. 

Definition  1.  An  n-dimensional  control  hybrid  systems  H  is  a  6-tuple 

H  =  (1) 

where 

Q  =  ,l</i:<oo,  isa  set  of  discrete  states  (which  are  called  control 

locations)', 

V  =  {Di,  i  €  Q,Di  c  is  a  collection  of  domains  of  H; 

S  =  Sa  U  Sc  is  a  collection  of  autonomous  and  controlled  switching  surfaces’, 

^  =  'J^a  U  T^c  is  a  collection  of  autonomous  and  control  resets. 

E  ~  EciM^d  is  the  set  of  admissible  continuous  and  discrete  controls; 

i  €  Q,  fi  :  Di  X  R”}  is  a  collection  of  control  vector  fields  as¬ 

signed  to  each  location; 

□ 

Each  of  these  components  shall  be  further  specified  in  the  next  part  of  the 
section. 

The  collections  of  autonomous  swithching  surfaces  (called  guards)  and  au¬ 
tonomous  resets 


{i,j)eEa} 
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where  Ea  C  Q  x  Q,  diVe  such  that  each  guard  is  a  subset  of  Di  and  each 
autonomous  reset  is  a  continuous  injective  map  acting  from  to  Dj. 
Similarly,  for  controlled  switching  surfaces  and  resets  we  have: 

5c  =  {Si^;  {ij)  e  Ec}  7^c  -  {ij)  6  E,}, 

where  Ec  C  Q  x  Q,  each  controlled  switching  surface  is  a  subset  of  Di,  and 
each  controlled  reset  is  a  continuous  injective  map  acting  from  to  Dj. 

The  set  of  discrete  controls  Ed  is  taken  to  be  {cij]  (z,y)  e  Ec},  where  each 
aij  is  a  discrete  control  that  can  be  applied  at  (and  only  at)  states  x  e  S]? . 

Take  an  arbitrary  initial  state  (i,  xo)  which  does  not  lie  on  any  of  the  switch¬ 
ing  surfaces.  Then,  for  any  control  u  e  Ec,  the  systems  evolves  according  to  the 
ODE 

x  =  fi{x,u),  x(0)=xo 

until  it  hits  (at  some  point  x)  either  (i)  a  guard  or  (ii)  a  controlled  switching 
surface  5*^. 

In  the  former  case  (i),  the  system  necessarily  switches  to  the  discrete  location 
j  and  the  continuous  component  of  the  states  resets  to  R^J{x).  Next,  the  system 
evolves  according  to  the  dynamics  fj  in  the  domain  Dj. 

In  the  latter  case  (ii),  we  distinguish  two  possibilities. 

(ii.a)  The  discrete  control  dik  is  applied  at  x;  then  the  system  switches  to  the 
location  k  and  the  continuous  component  of  the  state  resets  to  Rf{x). 
Next,  the  system  evolves  according  to  fk  in  Dk- 
(ii.b)  The  discrete  control  dik  is  not  applied;  the  system  continues  evolving  ac¬ 
cording  to  fi  in  Di. 

The  following  definition  of  a  hybrid  time  trajectory  is  based  on  [10,11]. 

Definition  2  (Forward  Hybrid  Time  Trajectory). 

A  (forward )  hybrid  time  trajectory  is  a  sequence  of  semi-closed  intervals 

r  =  {[Ti,Ti+i);  1  <i  <  N  <oo,Ti  <  Xi+i}. 

We  shall  use  the  symbol  N{t)  to  denote  the  size  of  the  time  trajectory  (i.e.  the 
number  of  semi-intervals  in  the  sequence  r),  the  symbol  (r)  to  denote  the  set 
{1, 2,  -  •  ■ ,  iV'(T)},  and  the  symbol  Too  to  denote  the  execution  time,  which,  for  a 
finite  N{t),  is  defined  to  be  Too  A  —'^x- 

Based  on  the  above  description  of  the  evolution  of  H,  for  any  control  pair  (u,g), 
where  it  is  a  continuous  control  in  Ec  and  (j  is  a  sequence  of  discrete  controls 
{vi,V2,'  • '  ,Vk\  Vi  €  Ed},  we  can  define  the  notion  of  the  control  execution 
=  {T,q,  (f)}  of  H  starting  at  the  initial  state  p  e  D,  where 

(i)  T  is  a  hybrid  time  trajectory  that  contains  the  sequence  of  the  switching 
times; 

(ii)  ^  :  (r)  Q  is  a  map  that  contains  the  sequence  of  discrete  locations  visited 
by  the  hybrid  trajectory; 
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(iii)  (f)  =  {4*j',j  t  ('?■)}  is  the  collection  of  continuously  differentiable  maps  of 
t  that  satisfies  the  corresponding  ODEs  and  the  switching  conditions  as 
described  above. 

As  in  [14],  we  shall  restrict  ourselves  to  the  study  of  hybrid  systems  that  are 
subject  to  the  following  assumptions. 

A1  The  control  hybrid  system  Hw  is  deterministic  and  non-blocking,  for  any 
control  pair  w  =  (w,  cr). 

A2  For  each  i  e  Di  is  assumed  to  be  a  non-empty,  closed,  contractible  n- 
dimensional  sub-manifold  of  with  a  piecewise  smooth  boundary. 

A3  For  each  e  e  Ea  and  e  e  Ec,  the  guard  S'®  and  the  controlled  switching 
surface  SJ  are  closed  (n  —  1) -dimensional  submanifolds  with  a  piecewise 
smooth  boundary.  These  sets  have  finite  number  of  connected  components. 
A4  All  resets  maps  are  continuous  and  injective. 

A5  None  of  the  autonomous  transition  sets  (i.e.  {S®,i2J(S®);  e  e  Ea};  de¬ 
noted  ATrans)  have  intersections  with  the  controlled  transition  sets  (i.e, 
{‘S'c?  6  e  Ec};  denoted  CTrans).  Further,  for  any  two  (autonomous 

or  controlled)  transition  sets  Bi,B2  (denoted  Trans),  we  have 

5i  n  ^2  0  ^  =B2  = 

for  some  i,ji,j2  e  Q. 

Remark  1.  We  note  that  the  restriction  5®  n  S'f  =  0  comes  from  the  fact  that 
H  is  assumed  to  be  deterministic.  The  rest  of  the  restrictions  of  A5  can  be 
somewhat  relaxed.  We  impose  A5  to  avoid  cumbersome  technical  details,  while 
illustrating  the  point  that  certain  hybrid  systems  can  be  represented  as  manifolds 
(termed  hybrifolds),  and  thus,  results  on  the  global  controllability  formulated  for 
manifolds  can  be  transformed  to  hybrid  systems.  □ 

Next  we  list  the  assumptions  on  the  continuous  part  of  the  dynamics  of  H, 

B1  For  each  i  e  Q,  Xi  e  C^(Di  x  C/;R^),  r  €  {1,2,  •  •  •  ,oo,a;},  where  denotes 
the  class  of  analytic  functions. 

B2  The  set  of  admissible  control  functions 

Ec  —  i^c^(R;  s  e  (1, 2,  •  •  • ,  oo}, 

is  the  set  of  all  R"^’" -valued  bounded  piecewise  C^(R;  R”’")  functions  of  time 
with  limits  from  the  right.  Hence  any  u  e  E^  defined  on  some  [ri,72), 
T2  <  00,  is  on  [Ti,T2)  with  the  exception  of  a  finite  number  of  points. 

For  the  results  formulated  in  this  paper  we  shall  need  r  =  1,  s  =  1. 

Definition  3.  A  control  hybrid  system  satisfying  assumptions  A1-A5  and  Bl- 
B2  is  called  a  regular  control  hybrid  system  with  controlled  and  autonomous 
switchings.  □ 
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Finally,  it  shall  be  assumed  that  the  system  H  is  non-Zeno  in  the  sense  that  in 
finite  time  only  a  finite  number  of  discrete  transitions  may  be  generated. 

Lemma  1.  Let  be  a  regular  control  hybrid  system.  For  any  control  pair  (u,  a) 
and  any  pe  D,  there  exists  a  unique  control  execution  of  H  starting  at  p.  □ 

3  Controlled  Hybrifold 

In  [14],  a  set  Mh  (called  the  hybrifold)  is  constructed  from  a  hybrid  system  with 
autonomous  switchings  H,  In  this  section  we  generalize  this  procedure  to  hybrid 
systems  with  autonomous  and  controlled  switchings,  prove  that  the  resulting  set 
M//  is  a  manifold  and,  finally,  define  the  controlled  hybrid  flow  on  M//. 

The  basic  idea  in  the  construction  of  the  hybrifold  is  to  glue  together  each 
switching  surface  to  the  image  of  the  corresponding  reset  map  by  identifying  any 
state  p  e  where  e  e  Eg,  s  =  a,c,  with  the  corresponding  image  R^ip)-  So  an 

IIQII 

equivalence  relation  ~  on  D  ^  is  generated  by 

i=l 

p^  RUp), 

for  all  ee  Es  and  p  e  This  relation  gives  rise  to  the  quotient  space 


Mh  =  D/ 


where  each  equivalence  class  is  collapsed  to  a  point. 

Let  TT  be  the  natural  projection  map 

TT :  Z)  — >  Mh 

which  assigns  to  each  p  its  equivalence  class.  We  put  the  quotient  topology 
on  Mh,  i.e.  the  smallest  topology  in  which  V  c  Mh  is  open  if  and  only  if 
C  D  IS  open  (in  the  relative  topology  of  D). 

Definition  4.  The  set  Mh  with  the  quotient  topology  defined  on  it  is  called 
the  controlled  hybrifold  associated  with  H. 

The  following  result  is  based  on  [14]. 

Theorem  1.  Mh  is  a  topological  n-manifold  with  boundary.  D 

Henceforth  we  shall  deal  not  with  the  original  domains  Di  but  rather  with  the 
hybrifold  Mh-  We  shall  assume,  without  loss  of  generality,  that  Mh  is  embedded 
in  for  some  n  <  m  <  oo. 

Definition  5  (Hybrid  Control  Flow).  Take  an  arbitrary  continuous  control 
ue  Ec  defined  on  some  [Ti,T2),  T2  <  oo,  a  sequence  of  discrete  controls  a,  and 
a  state  x  e  Mh-  Let  p  e  7r~^{x). 
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As  follows  from  Lemma  1,  there  exists  a  unique  control  execution  x  = 
of  H  starting  at  p  which  corresponds  to  the  control  pair  {u,a). 

We  shall  use  the  symbol  t  e  [Ti,T2),  to  denote  the  controlled 

hybrid  flow  on  Mh-  x,u,a)  is  defined  as  follows: 

^^{t,x,u,a)  A7r{(t)i{t)),  for  any  i  e  (r)  and  t  e  [Ti.n^i), 

In  particular,  we  have  ^^{ri,x,u,a)  =  7r(0i(ri))  =  7r(p)  =  x,  □ 


Remark  2.  We  note  that,  as  follows  from  the  Assumption  A5,  the  definition  of 
the  control  flow  on  Mh  does  not  depend  on  the  choice  of  the  representative  p  in 
the  equivalence  class  x.  □ 


Lemma  2.  For  any  control  u,  the  controlled  hybrid  flow  u,  ^r)  is  con¬ 

tinuous  on  Mh  with  respect  to  the  argument  t. 

Proof:  This  follows  from  the  fact  that  all  points  of  discontinuity  of  the  control 
hybrid  execution  are  removed  by  identifying  them  with  their  images  under  the 
corresponding  reset  maps.  □ 

4  The  Global  Controllability  of  Hybrid  Systems 

Let  H  be  an  arbitrary  regular  control  hybrid  system  and  Mh  its  controlled 
hybrifold.  In  this  section  we  relate  the  global  controllability  of  the  total  domain 
D  oi  H  with  the  global  controllability  of  Mh- 

Definition  6  (Accessible  sets  of  the  control  hybrid  system  H). 

Let  pe  D.  We  shall  say  that  a  state  p'  e  D  is  accessible  from  p  (with  respect  to 
V  C  DJ  if  there  exists  a  continuous  control  u  e  Ec,  defined  on  some  [Ti,T2), 
T2  <  00,  and  a  sequence  of  discrete  controls  a  =  {i^i,  *  •  •  such  that  the 
corresponding  control  execution  x  =  (r,  g,  (j))  of  H  starting  at  p  satisfies 

(i)  4>N{t){T)  =p\  for  some  T  e  [r7V(r);rjv{T)+i);  and 
(ii  for  any  f  e  (r)  and  t  e  [tj-;  Tj+i),  e  V. 

The  set  of  all  states  in  D  accessible  from  p  (with  respect  to  V)  shall  be  denoted 
by  A^(p).  In  the  case  V  =  D,  we  shall  write  Ad{p)-  □ 

Thus  we  assumed  that  an  accessible  state  p'  can  be  reached  from  p  in  finite  time 
using  a  finite  number  of  switching  (or  jumps)  between  control  locations. 

Remark  3.  We  observe  that,  as  follows  from  the  definition  of  the  control  execu¬ 
tion  of  iJ,  Rl{p)  e  Ad{p),  for  any  state  pe  e  e  Es,  s  =  a,c.  □ 

Similarly,  we  can  define  the  accessible  states  using  the  dynamics  of  the  controlled 
hybrifold  Mh- 
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Definition  7  (Accessible  sets  of  the  controlled  hybrifold  Mh)- 

Let  X  e  Mh  C  We  shall  say  that  a  state  x'  e  Mh  is  accessible  from  x  (with 
respect  to  V  C  Mh )  if  there  exists  a  continuous  control  u  e  Ec  defined  on  some 
[Ti,  r2),  T2  <  00,  and  a  sequence  of  discrete  controls  cr  =  {ui,  •  •  • ,  such  that 

(i)  x'  —  {T,x^u,a),  for  some  T  e  [ri,T2);  and 

(ii)  for  any  Ti  <t  <T,  ^^{t,  x,u,a)  eV. 

The  set  of  all  states  in  Mh  accessible  from  x  (with  respect  to  V)  shall  be  denoted 
by  A^{x).  In  the  case  V  =  Mh,  we  shall  write  A{x).  □ 

The  set  of  all  states  co-accessible  to  p  (to  x),  with  respect  to  C  D  (with  respect 
to  V  G  Mh),  H  (in  Mh)  is  defined  dually  and  shall  be  denoted  as  CA^{p) 
(as  CA^{x)). 

Remark  We  observe  that  for  any  p  e  D  and  any  neighborhood  V  of  p  in  D, 
we  have 

7r(A^(p))  C  (2) 

where  tt  :  D  — >  Mh  is  the  natural  projection  map.  This  is  because  any  orbit  in 
D  is  projected  by  tt  onto  an  orbit  in  Mh- 

On  the  other  hand,  let  p,p'  e  D  and  let  n{p')  e  A^(ir{p)).  Then  there  exist  some 
y,y'  eD  such  that  (i)  p-^y,p'  -^y'  and  (ii)  y'  e  Ap  (p).  In  other  words,  the 
existence  of  a  trajectory  from  7r(p)  to  7r(p^)  in  Mh  does  not  necessarily  imply  the 
existence  of  a  control  execution  connecting  p  to  p';  it  only  implies  the  existence 
of  a  control  execution  from  some  y  e  D  to  some  y'  e  D,  where  p  ~  p  and  y'  ^  p'. 

This  is  particularly  easy  to  see  in  the  situation,  where  at  some  controlled 
switching  surface  5^  at  least  two  discrete  controlled  can  be  applied. 

Take  x  e  S]?  and  consider  pi  =  and  p2  =  Then  x,pi,p2  lie  in 

the  same  equivalence  class  (they  are  glued  together  in  Mh)  and,  hence,  7r(pi) 
and  7r(p2)  are  mutually  accessible  in  Mh-  At  the  same  time  yi  and  p2  are  not 
necessarily  mutually  accessible  in  D. 

Hence  in  general,  we  do  not  have  the  reverse  to  (2)  inclusion  and  we  can  only 
guarantee  that  for  any  x  e  Mh  and  V  C  Mh, 

A^{x)cnl  U  Al^^^\p)\.  (3) 

□ 


Definition  8.  We  say  that  a  set  Di  C  D  is  controllable  with  respect  to  D2  C  D 
for  the  control  hybrid  system  H  if  A^'^{p)  =  Di,  for  all  pe  Di. 

In  the  particular  case  when  Di  =  D2  =  D,  and  Ad{p)  —  D,  for  all  p  e  D,  we 
shall  say  that  the  total  domain  D  is  globally  controllable  for  H, 

Similarly,  we  shall  say  that  a  set  Ci  C  Mh  is  controllable  with  respect  to  C2  C 
Mh  if  A^^{x)  =  Cl,  for  all  x  e  Ci.  Mh  is  globally  controllable  if  A(x)  =  Mh, 
for  all  X  €  Mh-  ^ 
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Theorem  2.  Let  H  he  a  regular  control  hybrid  system.  Then  the  total  domain 
D  is  globally  controllable  if  and  only  if  the  associated  hybrifold  M//  is  globally 
controllable. 

Proof: 

=>  Let  D  be  globally  controllable.  Then,  using  Remark  4  (2),  we  obtain  for  any 

xeMn, 


Mh  =  7r(Z))  =  7T{ADip))  C  =  A{x)  c  Mh, 

where  p  is  an  arbitrary  point  in  the  set  7r“^(a;)  C  D.  Hence  A{x)  ^  Mh,  for  any 
X  e  Mh,  and  Mh  is  globally  controllable. 

Conversely,  let  Mh  be  globally  controllable.  Take  any  p,p'  e  D.  Each  of 
them  could  lie  in  any  of  the  sets 


CTrans,  AT  vans,  DAD  —  Trans, 
i.e.  there  are  9  possible  cases. 

Consider,  for  instance,  the  case  when  p  e  Rl{Sl)  and  p'  e  for  some 

e  =  =  (z,/)  e  Ec.  Take  the  inverse  image  y'  =  {Ri}~^{p').  As  follows 

from  the  description  of  the  hybrid  executions  given  in  Section  2,  there  exist 
states  z  €  Dj  H  D  and  z'  e  Di  n  D  such  that  z  is  accessible  from  y  and  z^  is 
cp-accessible  to  y' .  Next  note,  that  since  z,z'  e  D  —  Trans  and  tt  is  1  to  1  on 
D,  from  the  existence  of  an  orbit  connecting  7r(z)  to  7r(z')  in  Mh  follows  the 
existence  of  a  control  execution  that  drives  z  to  z\  Finally,  combining  all  the 
accessibility  relations  for  p,  z,  z',y',p'  we  conclude  that  p'  e  Ad{p). 

The  rest  of  the  cases  can  be  considered  in  an  analogous  manner.  Thus 
Ad{p)  =  D,  for  any  peD,  and  D  is  globally  controllable.  □ 

The  above  result  allows  us  to  use  the  hybrifold  and  the  continuous  controlled 
hybrid  flow  defined  on  it  in  order  to  study  the  global  controllability  of  the  orig¬ 
inal  control  hybrid  system.  The  advantage  of  this  approach  is  in  the  fact  that 
the  controllability  results  formulated  for  differential  control  systems  acting  on 
subsets  or  sub-manifolds  of  can  be  transformed  to  control  hybrid  systems. 
This  shall  be  demonstrated  in  the  next  section. 

5  Hybrid  Fountains 

In  this  section  we  introduce  the  notion  of  a  hybrid  fountain  which  we  shall  use  as 
the  main  hypothesis  in  our  controllability  result.  Henceforth  the  symbol  Bs(x), 
where  x  e  Mh,  0  <  (^  e  shall  denote  the  m-dimensional  ball  with  the  center 
a:  and  the  radius  5.  The  sets  A^^^P^p)  and  CA^^^P\p)  shall  be  denoted  as  A^{p) 
and  CA^{p),  respectively. 

Definition  9.  A  state  x  e  Mh  is  called  a  hybrid  fountain  if 

3p>0yS,  0<S<fi,  A\x)  -  {x}  and  CA^(x)  -  {x} 
are  non-empty,  open  sets. 


(4) 
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If  the  function  p  A  sup{/i;  such  that  the  condition  (4)  holds}  is  continuous  at 
X,  we  shall  say  that  x  is  a  continuous  hybrid  fountain.  If  p  is  unbounded  at  x  we 
consider  it  to  be  continuous  at  x.  ^ 

The  reader  is  referred  to  [2,3,7]  for  applications  of  the  fountain  condition  to  the 
study  of  ordinary  differential  systems  acting  on  subsets  of  See  also  [8],  where 
a  set  of  algebraic  conditions  for  verification  of  the  fountain  property  is  presented, 
and  [4]  where  applications  to  hierarchical  hybrid  control  theory  are  outlined. 

Henceforth  we  shall  use  the  term  controlled  closed  orbit  in  the  sense  of  con¬ 
trolled  loop. 

Theorem  3.  Let  each  x  e  Mh  be  a  continuous  hybrid  fountain  and  let  for 
each  X  e  Mh  there  exist  a  control  u  e  Ec  such  that  x  lies  on  a  nontrivial 
(controlled  under  u)  closed  orbit  in  Mh-  Then  each  connected  component  of 
[MhY  is  controllable  with  respect  to  Mh- 

Proof:  Let  C  denote  one  of  (the  finite  number  of)  the  connected  components 
of  [Mh]°-  For  any  two  states  x,x'  in  C  we  define  a  relation  in  such  a  way 
that  X  if  3^nd  only  if  there  exists  a  (controlled)  nontrivial  closed  orbit  in 

Mh  passing  through  both  x  and  x',  i.e.  there  exists  a  control  pair  u,  a  defined 
on  some  [Ti,T2),  T2  <00,  such  that 

(i)  3r,  ri<r<T2,  ^^(ri,x,u,(7) -iZ^(T,x,u,(t);  and 

(ii)  3  t,  Ti  <  t  <  T,  ^{t,p,u,a)  =p'. 

Clearly,  the  relation  is  reflexive  (since  each  state  in  Mh  lies  on  a  nontrivial 
orbit),  symmetric  and  transitive.  Hence  there  exists  a  partition  of  C  on  the 
equivalence  classes  of  Let  [x],  for  an  arbitrary  x  e  C,  denote  the  equivalence 
class  containing  x.  We  claim  that  [x]  is  an  open  subset  in  C. 

Indeed,  take  any  2  e  [xj.  Let  u  and  0  <  t  <  00  be  such  that  2  = 

Define  a  =  ^{t  -  Z^,x,w,a)  and  b  =  ^{t -\r  A,x,u,a),  A  >  0.  Then,  since  a 
and  b  are  hybrid  fountains,  the  sets  A^{a)  —  {a}  and  CA^{b)  —  {6}  are  open,  for 
sufficiently  small  ^  >  0.  Choose  A  so  small  that  2  e  A^{a)  and  2  e  CA^{b)  (this 
is  possible  since  a,  b  are  continuous  hybrid  fountains).  Then  there  exists  an  open 
neighborhood  A^(2)  of  2  which  lie  in  the  intersection  {A^{a)  -  {a})  n  {CA^{b)  - 
{6}).  Each  state  2'  e  N{z)  is  accessible  from  a  and  co-accessible  to  6.  Moreover, 
since  a,b  e  [x],  we  conclude  that  2'  lies  on  a  non-trivial  orbit  passing  through  x. 
This  is  true  for  all  z'  e  N{z),  hence  N{z)  C  [x]  and  [x]  is  open,  as  claimed. 

For  any  x,  x'  e  C  we  have  [x]  D  [x'j  0  [x]  =  [x'j,  so  any  two  equivalence 

classes  are  either  disjoint  or  coincide.  Thus  the  set  C  can  be  represented  as  the 
disjoint  union  C  =  AU  B,  where  yl ^  [x],  for  some  x  e  C,  and  B  A  [J  [x'j. 

x'  f.C 

A  and  B  are  open  and  disjoint.  Since  C  is  connected,  we  conclude  that  B  is 
empty,  i.e.  any  x'  e  C  is  such  that  x  ~o  -  In  other  words,  any  two  states  in 
C  lie  on  a  nontrivial  controlled  orbit  in  Mh  and  hence,  C  is  controllable  with 
respect  to  Mh-  ^ 
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Remark  5.  We  note  at  this  point  that  weaker  recurrence  conditions  can  be  used 
instead  of  the  existence  of  closed  orbits.  Also,  for  the  proof  of  the  above  result, 
the  continuous  hybrid  fountain  condition  (4)  can  be  relaxed  to 

p{^)  i4sup{//  >  —  {x},CA^{x)  —  {x}  are  non-empty,  open  sets} 

is  continuous,  for  all  x  e  Mh-  □ 


Theorem  4.  Assume  that  the  hybrifold  Mh  is  connected  and  the  conditions  of 
Theorem  3  are  satisfied.  Then  Mh  is  globally  controllable. 

Proof:  As  has  been  shown  in  [14],  Mh  is  n-dimensional  manifold  (possibly  with 
boundary).  This  implies,  by  definition,  that  for  any  boundary  state  in  dMn 
there  exists  a  neighborhood  which  is  homeomorphic  to  R!}.  Hence  [Mh]°  and 
Mh  have  the  same  number  of  connected  components;  in  particular,  [Mh]°  is 
connected  if  and  only  if  Mh  is  connected. 

Take  any  boundary  state  x  €  dMn.  Then,  since  a:  is  a  hybrid  fountain,  the 
sets  A  (a:)  —  {a;}  and  CA^{x)  —  {a:}  are  non-empty  and  open,  for  sufficiently  small 
(^  >  0.  Hence  there  exist  a  e  {A\x)~{x})n[MH]^  and  b  e  {CA^{x)-{x})n[MHY . 

^  For  any  state  p  e  [Mh]°  we  can  find  a  control  u  e  X'c  which  would  drive  a 
p'  and  a  control  u'  e  which  would  drive  p'  to  6.  This  is  because  a,  b,p'  lie  in 
[Mh]''  and,  as  follows  from  Theorem  3,  [Mh]°  is  controllable.  We  conclude  that 
arbitrary  p  e  dMn  and  p'  e  [Mh]",  and  thus  arbitrary  p,p'  e  Mh,  are  mutually 
accessible.  Hence  Mh  is  globally  controllable.  □ 

Consider  the  directed  graph  T  of  H  which  has  vertices  Q  and  edges  E.  We  can 
treat  it  as  a  finite  state  machine,  by  defining  the  transition  function  ^  :  Q  ^  Q 
in  such  a  way  that  for  any  ij  e  Q,  ^(i)  =  j  if  and  only  if  (ij)  e  E  or  i=j. 

Theorem  5.  Assume  that  the  conditions  of  Theorem  3  are  satisfied.  Then  Mh 
is  globally  controllable  if  and  only  if  the  graph  E  —  {Q,  E}  is  controllable  as  a 
finite  state  machine. 

Proof: 

=>  Assume  that  Mh  is  globally  controllable.  Then  for  any  ij  e  Q,  i  ^ 
take  some  states  p  e  Di  and  p'  e  Dj.  There  exists  a  trajectory  ip  from  p  to  p' 
in  Mh.  Let  the  sequence  i  =  rj,  rs,  •  •  ■ ,  =  j,  £>  I,  he  such  that  ip  switches 

consecutively  from  the  domain  Dr^  to  the  domain  ,  where  s  =  1, 2,  •  •  • ,  ^-1, 
using  the  corresponding  guards  and  the  images  of  the  reset  maps.  Hence  each 
consecutive  pair  (r^,  r^+i)  belongs  to  E  and  hence,  there  exists  a  trajectory  from 
the  state  i  to  the  state  j  in  the  graph  F.  Since  this  holds  for  an  arbitrary  pair 
{h  j)  ^  Q,  we  conclude  that  E  is  controllable  as  a  finite  state  machine. 

4=  Conversely,  assume  that  E  is  controllable  as  a  finite  state  machine.  Then 
for  any  two  states  p,p'  e  D  take  i  and  j  such  that  p  e  Di  and  p'  e  Dp  If 
i  +  E  find  a  trajectory  i  =  ri,r2,  •  •  ■ ,  =  j,  ^  >  1,  in  the  graph  E.  Since  each 

consecutive  pair  (r5,r5^.i)  belongs  to  F,  there  exists  a  guard  in  the 

domain  Dy^  which  is  identified  with  the  image  of  the  reset  map  R{r^  r  +i)  in  the 
domain  Dy,^^.  Hence  the  domains  Dy^  and  Dy^_^^,  and  thus  A  and^A,  lie  in 
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one  connected  component  of  Mh-  This  can  be  shown  for  alli^jeQ.  Hence  Mh 
is  connected  and,  as  follows  from  Theorem  4,  Mh  is  globally  controllable.  □ 
An  application  of  the  obtained  results  can  be  illustrated  on  a  two  water  tank 
system  example,  which,  for  the  lack  of  space,  shall  be  described  briefly.  The  water 
can  be  added  to  the  system  at  some  rate  u;  >  0  (where  we  treat  the  parameter 
w  as  control)  in  two  different  modes: 

1:  the  water  is  added  (exclusively)  via  tank  1; 

2:  the  water  is  added  (exclusively)  via  tank  2. 

In  addition  to  that,  the  water  is  removed  from  tank  z,  i  =  1, 2,  at  some  constant 
rate  z;i  >  0.  The  two  tank  system  can  be  modeled  as  a  control  hybrid  system  in 
the  following  way.  We  shall  distinguish  two  control  locations  -  each  corresponds 
to  one  of  the  modes,  i.e.  Q  =  {1,2}.  The  continuous  dynamics  at  the  locations 
are  as: 

q  =  l:  I  ^  ~  (a:,  y)  e  Di  A  {[/i,  oo)  x  [^2,  oo)}, 

q  =  2:  (x,?/)  «  I>2  4  {[^i.oo)  X  [^2,oo)}, 

where  x,  y  denote  the  levels  of  water  in  the  tanks  1  and  2,  respectively. 

The  class  of  control  functions  is  taken  to  be  the  set  of  all  functions  taking  values 
in  and  satisfying  B2. 

The  guards  are  defined  as 

<^(1,2)  =  1  X  {(x,?/)  e  Di;  y  =  h},  =  2  x  {(x,?/)  e  D2;  x-  h}. 

The  resets  are  defined  in  such  a  way  that  when  hitting  a  guard  in  one  domain  the 
system  switches  to  the  other  control  location,  without  changing  the  continuous 
part  of  the  state,  i.e. 

-^(i,2)(l j ^5 ^2)  ~  (2;2:,^2),  'f^(2,l)(25  ^1, y)  =  (1,^1, ?/)• 

Furthermore,  assume  that  for  some  level  >  ^2,  in  the  first  tank,  a 

discrete  switching  to  the  second  tank  is  allowed. 

To  construct  the  corresponding  controlled  hybrifold  we  identify  (via  the  identity 
reset  maps)  the  x  =  h,  y  =  h,  y  =  I  axes  of  Di  with  the  x  =  h,  y  =  h,  y  =  I 
axes  of  D2,  respectively. 

Using  the  obtained  results,  it  can  be  verified  that  each  state  of  the  hybrifold 
is  a  hybrid  fountain  lying  on  a  closed  orbit.  Hence,  the  two  water  tank  system 
can  be  shown  to  be  globally  controllable. 

Remark  6.  In  conclusion  we  note  that  algebraic  conditions  for  verification  of  the 
fountain  property  at  each  state  x  e  Mh  shall  be  presented  in  a  future  version  of 
the  paper. 
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Abstract.  Models  of  industrial  processes  often  contain  discrete  phe¬ 
nomena  superimposed  on  the  continuous  system  behavior.  Simulation 
of  batch  processes,  start-up  and  shutdown  procedures,  fault  diagnosis 
and  alarms  fall  under  this  category.  Models  for  such  processes  require  a 
mathematical  framework  for  both  its  continuous  and  discrete  state  tran¬ 
sitions.  A  key  problem  in  hybrid  simulation  lies  in  the  detection  and  ex¬ 
act  location  of  discontinuities  that  delineate  state  changes.  Hence,  hybrid 
systems  require  special  numerical  procedures,  which  are  not  available  in 
conventional  integration  methods.  In  this  paper,  important  issues  per¬ 
taining  to  the  numerical  aspects  in  hybrid  simulation  will  be  discussed. 
We  will  demonstrate  a  new  approach  to  event  handling.  The  main  target 
of  this  new  approach  is  enhanced  computational  performance  without 
loss  of  rigor.  The  authors  anticipate  the  significance  of  high  speed  in  the 
advent  of  new  challenges  in  optimal  control  and  dynamic  optimization 
problems.  The  improvements  are  due  to  the  exploiting  local  monotonicity 
and  smooth  function  properties  observed  in  varaible  step-size  integration 
algorithms. 


1  Introduction 

The  mathematical  model  for  a  physical  process  expresses  mass,  energy  and  mo¬ 
mentum  balances  by  means  of  differential  equations.  These  solutions  to  the  con¬ 
servation  equations  lead  to  continuous  trajectories  of  the  state  variables.  In  in¬ 
dustrially  relevant  process  models,  however,  discrete  actions  or  discontinuities 
may  interrupt  the  continuous  evolution  of  state  variables.  As  an  example,  con¬ 
sider  the  cyclic  operation  of  a  batch  unit  under  logical  control.  Each  batch  cycle 
is  composed  of  different  stages  or  steps,  e.g.  fill,  heat,  react,  etc.  The  system 
dynamics  in  each  state  is  governed  by  differential  equations,  and  switching  con¬ 
ditions  that  cause  the  transition  from  one  stage  to  the  next.  The  state  transitions 
of  a  process  model  could  reflect  physical  discontinuities  such  as  hysteresis  or  sat¬ 
uration.  Otherwise,  they  may  be  externally  imposed  on  the  process  by  logical 
controller  actions  or  forcing  function.  Systems  with  discontinuities  superimposed 
on  the  continuous  system  behavior  are  termed  continuous-discrete  or  hybrid  sys¬ 
tems.  The  dynamics  of  hybrid  systems  falls  between  two  extremes:  (i)  Systems 
are  driven  by  continuous  dynamics,  if  the  number  of  discontinuities  is  small, 
e.g.  batch  operations,  (ii)  In  event-driven  systems,  discontinuities  dominate  the 
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process  dynamics,  e.g.  digital  control  system  (DCS).  Hence,  continuous  and  the 
discrete  sub-models  could  be  viewed  as  either  super-systems  or  sub-systems  of 
each  other.  Standard  numerical  treatment  of  hybrid  systems  via  ’continuous’ 
integration  methods  breaks  down  at  sufficiently  abrupt  discontinuities.  Equally, 
discrete  simulation  techniques  such  as  Petri-Nets  fail  to  express  continuous  dy¬ 
namics.  Hence,  a  mathematical  framework  for  addressing  both  continuous  as 
well  as  discrete  process  dynamics  effectively  is  needed.  Recently,  new  challenges 
in  optimal  control  and  dynamic  optimization  involve  discrete-continuous  models 
for  constraints  in  non-linear  optimization  functions  [1].  The  sensitivity  function 
evaluations  of  gradient-based  search  techniques  require  repeated  calls  to  the  hy¬ 
brid  system  equations.  In  the  light  of  these  repeated  computations,  the  efficiency 
of  the  hybrid  algorithm  is  crucial  [2]. 

A  concise  presentation  of  effective  algorithms  for  hybrid  systems  simulation, 
their  strengths  and  weaknesses  is  the  main  goal  of  this  article.  Section  2  reviews 
and  assesses  prior  work  in  hybrid  simulation.  Section  3  develops  the  mathemat¬ 
ical  framework  for  a  new  approach  to  efficient  continuous-discrete  simulation.  It 
will  present  a  hierarchical  procedure  based  on  statistical  observations,  yielding 
a  desired  performance  increase.  Section  4  discusses  some  of  the  advanced  topic 
issues  in  hybrid  simulation,  e.g.  multiple  and  simultaneous  discontinuities  in  an 
interval.  It  also  analyzes  step-size  control  and  recommends  solver  tolerances  for 
the  algorithm.  Finally,  an  application  in  section  5  quantifies  the  performance  of 
the  new  algorithm  using  benchmark  case  studies,  and  compares  the  proposed 
methodology  with  existing  approaches. 

2  Background 

A  hybrid  system  is  characterized  by  three  important  elements  (i)  a  continuous 
part  (ii)  a  discrete  part  and  (iii)  state  transitions  [3].  The  continuous  dynamics 
of  a  physical  process  can  be  modeled  by  sets  of  differential  equations.  Transient 
balance  equations  lead  to  time-dependent  trajectories  of  the  state  variables,  i.e. 
the  continuous  part.  The  evolution  of  state  variables  may  be  interrupted  by  a 
discrete-time  discontinuity  called  ’events'  ,  i.e.  the  discrete  part.  Events  may 
involve  discontinuous  changes  in  the  state  variable  values  or  their  derivatives, 
a  switch  in  the  underlying  model  equation  or  both.  After  an  event,  the  system 
traverses  into  a  new  state,  i.e.  state  transition. 

Each  state  of  a  continuous-discrete  process  can  be  associated  to  distinct 
mode,  The  ’continuous’  dynamics  of  each  mode,  involves  a  set  of  differ¬ 
ential  equations  given  by  equation  (1). 


2/?  2)7  ^(^))  ^)  ^n+l]  (1) 

(1)3  X  X  X  X  i?  ^  vector  function; 

X  e  and  y  e  denote  the  algebraic  and  differential  variables  respectively. 
u{t)  e  are  the  known  system  inputs.  The  number  and  the  type  of  model 
equations,  f^,  are  specific  to  the  current  mode 
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The  cause  for  a  transition  into  a  new  mode  can  be  expressed  mathematically 
by  an  event  function.  More  precisely  a  switch  occurs  when  the  function, 
crosses  a  threshold  value  of  zero.  The  customary  normalization  to  the  right- 
hand-side  zero  explains  its  alias  as  z-function  as  given  by  equation  (2). 


=  0  (2) 

In  equation  (2),  is  termed  the  event  function  associated  with  the  transi¬ 
tion  of  mode  $  to  mode  It  is  a  function  of  the  state  variable  and  the  indepen¬ 
dent  variable  time.  The  state  transition  takes  place  at  the  exact  time  instant, 
r,  at  which  the  conditional,  2:^^,  becomes  zero.  This  time,  f*,  terminates  the 
validity  of  the  old  mode  The  new  mode  “0  starts  exactly  at  this  same  instant, 
giving  rise  to  two  sets  of  state  variables  values  associated  with  time  t*.  Each  set 
corresponds  to  one  mode  $  and  0  respectively. 

In  principle,  hybrid  systems  of  equation  (1)  and  switching  function  (2)  can 
be  numerically  integrated  via  integration  routines  combined  with  logical  if- 
statements  to  check  for  transitions  [4].  Gear  [5],  has  shown  that  such  a  brute  force 
approach  with  multi-step  integration  methods  leads  to  gross  losses  in  speed  and 
accuracy.  In  the  worst  case,  discontinuities  may  cause  floating-point  errors  and  a 
subsequent  crash  of  the  solver  algorithm.  A  robust  hybrid  simulation  algorithm 
should  first  identify  whether  events  have  occurred,  locate  their  exact  time,  and 
execute  the  appropriate  actions  pertaining  to  the  event.  This  approach  usually 
involves  a  technique  called  discontinuity  locking. 

2.1  Discontinuity  Locking 

Multi-step  integration  routines  solve  systems  of  differential  equations  via  re¬ 
peated  computations  executed  in  small  intervals  with  step-size  h.  In  disconti¬ 
nuity  locking,  the  validity  of  current  state  ^  is  enforced  throughout  the  entire 
length  of  the  current  integration  step  with  a  small  step-size.  Hence,  the  state 
variable  trajectories  are  computed  smoothly  throughout  the  small  interval  using 
equations  (1).  Then,  the  trajectories  of  z-functions  (equation  2)  are  examined  for 
any  possible  zero-crossings  in  the  current  interval.  This  first  phase  is  called  event 
detection.  A  zero  penetration,  which  must  necessarily  lie  within  the  bounds  of 
the  current  interval,  h,  indicate  the  occurrence  of  an  event.  This  changeover  is 
triggered  for  a  zero  penetration  from  both  the  negative  or  the  positive  side.  The 
event  with  the  earliest  zero-crossing  is  called  the  active  event.  The  precise 
value  of  this  event  time,  t*,  the  corresponding  values  of  all  state  variable  values 
must  be  computed  by  adequate  means  such  as  interpolation.  This  second  phase 
is  termed  event  location.  After  locating  the  event  time,  the  system  switches  into 
the  new  state  as  directed  by  the  actions  associated  with  the  state  transition. 
Possible  conflicts  among  multiple  and  competing  events  have  to  be  handled  here 
also.  Therefore  this  last  stage  is  called  step  completion. 

Discintinuity  locking  requires  the  ability  to  extrapolate  into  the  undefined 
region  smoothly.  Therefore  discontinuity  locking  may  fail  for  systems  bordering 
a  numerical  singularity  such  as  logarithmic  functions.  Nevertheless,  this  approach 
has  been  incorporated  in  most  of  the  algorithms  ([6],  [7],  [8]). 
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2.2  Review  of  Existing  Algorithms 

Several  algorithms  have  been  developed  earlier  for  detecting  and  locating  discon¬ 
tinuities  in  a  dynamic  simulation  problem.  The  algorithms  vary  in  their  approach 
to  the  event  detection  and  location  phases. 

Carver  [6]  developed  an  event-handling  algorithm  for  systems  of  ordinary 
differential  equations.  State  transitions  were  modeled  via  algebraic  discontinuity 
or  event  functions,  cf.  equation  (2).  Their  differentials  were  appended  to  the 
systems’  set  of  differential  equations.  The  combined  augmented  system  of  dif¬ 
ferential  equations  was  integrated  using  a  modified  Hindmarsh-Gear  method. 
Events  were  identified  by  tracking  sign  changes  of  the  event  function  in  each  in¬ 
tegration  step.  Events  were  located  by  solving  a  order  polynomial  for  a  zero 
crossing.  Hay  and  Griffin  [9]  used  a  similar  approach  based  on  an  augmented 
system  including  the  derivatives  of  the  discontinuity  functions,  and  sign  changes 
of  the  event  function.  For  event  location,  they  deployed  linear  and  quadratic 
interpolation  with  a  reduced  step  size. 

Joglekar  and  Reklaitis  [7]  detected  events  by  checking  for  threshold  crossing 
of  event  function.  The  event  time  was  found  by  solving  a  order  interpolation 
polynomial  by  means  of  a  Newton’s  iteration  scheme.  However,  their  approach 
did  not  explicitly  solve  for  the  exact  event  time  and  was  therefore  prone  to 
inaccuracy. 

Birta  et.  al  [10]  approximated  the  event  conditional  by  a  cubic  polynomial 
and  considered  all  possible  configurations  of  the  polynomial  for  event  detection. 
Events  were  located  by  a  Regula-Falsi  method  and  a  Newton’s  iteration  scheme. 
Shampine  et  al.  [11]  uses  a  Sturm  sequence  to  determine  the  zero  of  a  linear 
event  conditional  and  locates  the  event  time  by  using  a  bisection  method  in 
conjunction  with  a  Sturm  sequence. 

Pantelides  [12]  directly  integrated  the  algebraic  event  functions  alongside  the 
system  differential  equations.  Zero  crossings  in  the  trajectories  of  this  differential- 
algebraic  system  indicated  events.  Events  were  located  using  bisection  method, 
Preston  and  Berzins  [13]  developed  an  event-handling  algorithm  for  a  particular 
class  of  dynamic  simulation  problems  pertaining  to  valve  operations.  A  disconti¬ 
nuity  was  detected  through  the  use  of  a  switch  function  that  changed  sign  when 
the  valve  opens  or  closes.  To  find  the  time  at  which  a  discontinuity  occurred, 
backward  interpolation  on  the  switch  function  was  used. 

Park  [8]  developed  a  rigorous  event-handling  algorithm  with  superior  perfor¬ 
mance  of  the  root  exclusion  than  the  one  proposed  by  Shampine.  It  employed 
an  interval  arithmetic  technique  for  event  detection.  The  root  finding  procedure 
consisted  of  two  steps  (i)  a  root  exclusion  test  and  (ii)  Newton’s  method  with 
recursive  interval  bisection.  The  root  exclusion  test  used  interval  arithmetic  to 
obtain  an  interval  enclosure.  An  interval  enclosure  of  a  function  with  one  argu¬ 
ment  captures  the  largest  and  lowest  values  the  function  can  assume.  Functions 
with  enclosures  not  containing  zero  cannot  have  a  root  in  the  interval  of  interest. 
Enclosures  with  zero  may  or  may  not  exhibit  a  real  root.  Consequently,  an  in¬ 
terval  Newton  Method  combined  with  interval  bisection  was  deployed  to  analyze 
intervals  with  enclosures  containing  a  zero. 
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The  algorithms  discussed  above  broadly  fall  into  two  categories:  Type  I  algo¬ 
rithms  only  detect  events  that  have  a  single  zero  crossing  within  the  integration 
step.  Type  II  algorithms  can  identify  most  events  with  multiple  zero-crossings. 
The  first  category  of  algorithms  is  based  on  the  conventional  approach  to  event 
handling  and  detect  a  discontinuity  by  checking  for  a  sign  change  in  the  event 
function  ([6], [7], [9], [13]).  While  type  I  algorithms  are  fast,  they  are  unreliable 
since  they  may  miss  situations  with  multiple  roots  in  the  interval.  Many  of  the 
algorithms  of  this  category  also  suffer  from  a  phenomenon  termed  discontinuity 
sticking^  first  described  by  Park  and  Barton  [8].  Small  inaccuracies  in  the  event 
function  cause  repeated  firing  of  the  same  event  in  the  subsequent  integration 
steps.  This  undesirable  effect  is  due  to  renewed  zero  crossing  caused  by  small 
time  drifts  due  to  the  double  precision  arithmetic  of  event  location.  Neverthe¬ 
less,  they  are  adequate  for  modeling  physical  systems  with  linear  discontinuity 
functions. 

Type  II  algorithms  deploy  more  rigorous  root  exclusion  tests  for  detection  of 
discontinuities  ([8],  [11]).  The  root  exclusion  test  eliminates  state  variable  tra¬ 
jectories  without  zeros  in  the  interval.  The  algorithms  belonging  to  this  category 
consume  more  manipulations  than  type  I  algorithms.  In  [8],  a  root  exclusion  test 
is  based  on  a  Sturm  sequence  [8].  In  this  approach,  it  is  necessary  to  construct 
a  Sturm  sequence  for  an  order  polynomial  which  required  (n-|-l)(n+2)/2 
multiplications.  On  the  other  hand,  Barton’s  root  exclusion  test  uses  a  clev¬ 
erly  normalized  interval  arithmetic  {I A)  technique  requiring  just  n  evaluations 
[8].  However,  the  I  A  fails  to  identify  a  zero  with  multiplicity  greater  than  one, 
because  of  singularity  in  the  Interval  Jacobian  matrix.  Moreover,  the  interval 
arithmetic  is  typically  twice  as  expensive  as  conventional  algebra.  While  lA 
methods  are  excellent  for  systems  with  a  small  number  of  events,  they  may  not 
be  optimal  for  event-driven  systems. 

In  the  following  section  a  new  algorithm  with  a  more  efficient  root  exclusion 
test  is  presented.  A  hierarchical  approach  to  event  detection  based  on  the  sta¬ 
tistical  evidence  of  event  occurrence  will  be  discussed.  The  improvements  are 
mainly  targeted  at  improved  performance  as  required  in  the  context  of  dynamic 
optimization  ([1],[2]).  A  detailed  discussion  of  the  algorithm  follows. 

3  Hierarchical  Approach  to  Discontinuity  Handling  in 
Event-Driven  Processes 

The  main  thrust  of  the  new  algorithm  lies  in  providing  a  simple  and  yet  rig¬ 
orous  root  exclusion  test  for  high-order  numerical  multi-step  integrators  with 
adaptable  step-size  control.  Two  avenues  for  performance  improvements  will  be 
offered.  Numerical  experiments  show  that  within  an  integration  step,  most  vari¬ 
able  trajectories  are  locally  monotonic.  This  observation  can  be  attributed  to  the 
step-size  control  mechanism,  which  discards  trajectories  with  infliction  points  or 
non-smooth  behavior.  This  property  holds  specifically  true  for  most  event  con¬ 
ditionals  of  physical  processes.  A  second  issue  exploits  the  fact  that  in  most 
intervals  no  events  occur.  We  will  show  in  the  next  section  how  these  two  prop- 
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erties  can  be  used  advantageously  for  developing  an  efficient  hybrid  simulation 
algorithm. 

Fig.  1  outlines  the  information  flow  of  the  variable  step-size  integrator  with 
event-handling.  White  boxes  demarcate  the  stages  of  standard  high-order  in¬ 
tegrators;  grayed  fields  underscore  additional  steps  required  for  event  handling. 
The  event  handling  part  traverses  through  the  usual  three  stages:  (i)  event  detec¬ 
tion  (ii)  event  location  and  (iii)  step  completion.  The  objective  of  event  detection 
phase  aims  at  examining  whether  any  event  function  in  the  present  mode  had 
become  zero  in  the  current  integration  step.  Step  completion  executes  the  actions 
associated  with  the  state  transition. 


Fig.  1.  Information  flow  diagram  for  integration  with  event-handling 
3.1  Event  Detection 

For  event  detection,  the  event  functions  in  equation  (2)  are  examined.  Statistical 
observations  on  monotonicity  cis  well  as  events  frequency  led  us  to  conceive  a 
hierarchical  procedure  composed  of  three  layers  depicted  in  Figure  2.  The  top 
layer  handles  intervals  with  the  highest  likelihood  of  occurrence.  Lower  layers  are 
necessary  to  safeguard  rigor  with  increasing  effort.  Typically  the  lowest  nesting 
levels  are  only  reached  in  rare  occasions  such  as  the  tough  benchmark  case  studies 
in  section  5. 

Locally  monotonic  intervals.  Local  montonicity  follows  from  same-signed 
gradients  at  the  support  points  in  high-order  integrators.  In  case  of  a  fifth  order 
Runge-Kutta  (RK)  method  [14],  there  are  four  gradients  available  at  no  addi¬ 
tional  effort.  Hence,  the  montonicity  test  costs  but  a  simple  boolean  operations 


Modeling  of  Continuous-Discrete  Processes  393 


for  a  sign  change  in  the  first  derivatives.  In  most  integration  steps  no  events 
occur,  and  those  with  a  zero  crossing  are  likely  to  exhibit  monotonic  trajectory. 
For  locally  monotonic  event  functions,  a  sign-change  corresponding  to  the  be¬ 
ginning  and  the  end  of  the  integration  interval,  suffices  for  detecting  an  event. 
The  adverse  outcome  of  the  monotonicity  test  indicates  a  rare  non-monotonic 
interval. 


Fig.  2.  Information  flow  diagram  for  hierarchical  event  detection 

Locally  non-monotonic  intervals.  For  locally  non-monotonic  trajectories  the 
second  level  of  Fig.  2  is  reached.  We  are  especially  interested  in  cheaply  excluding 
non-monotonic  intervals  without  zeros  to  avoid  rigorous  root  search.  For  most 
intervals,  a  simple  over-estimator  or  under-estimator  to  the  event  function  is 
adequate,  see  figure  3  (i)  and  3  (ii).  The  first  order  estimator,  2°"^,  requires  the 
initial  function  and  its  gradients  information  as  indicated  by  equation  (3).  Note 
that  this  test  applies  only  for  non-monotonic  intervals  where  there  is  at  least 
one  zero  in  the  first  order.  Therefore,  the  interval  has  to  exhibit  at  least  one 
extremal  point.  Hence  should  enclose  the  maxima  or  minima  (see  figure  3(i) 
and  3(ii)) 

=  z"  +  (i  X  h)  (3) 

In  the  above  equation,  is  the  event  conditional  at  the  initial  point  in  the 
interval,  z  its  derivative,  and  h  is  current  step  size.  Figure  3(iii)  shows  an  instance 
in  which  the  over/ under-estimator  envelope  fails.  Although  these  situations  are 
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possible  in  theory,  the  step-size  control  mechanism  of  figure  1  rejects  such  trajec¬ 
tories.  A  formal  proof  of  this  property  based  on  principles  of  flexibility  analysis 
is  being  developed,  and  will  be  discusses  in  [16]. 


Fig.  3.  estimators  to  the  true  function  (i)  under-estimator  (ii)  over-estimator  to  the 
function  and  (iii)  case  where  root  exclusion  fails 

Simple  Root  Exclusion  Test.  The  simple  root  exclusion  test,  i.e.  >  0, 
excludes  the  possibility  of  zero  crossing  in  an  integration  step.  In  this  case,  the 
integration  is  continued  without  a  state  transition. 

Advanced  Root  Exclusion  Test.  and  z^  lying  on  opposite  sides  of  the 
abscissa,  i.e.  <  0  ,  indicates  a  necessary,  but  not  a  sufficient  criterion  for 

an  event.  This  branch  leads  to  level  three  analysis  in  figure  2.  Two  stages  are 
involved: 

Examine  Support  Points:  In  each  integration  step,  the  discrete  support 
points,  =  2,3,  ..,,5  are  examined  for  a  sign  change.  If  two  support  points 
have  an  opposite  sign,  then  a  zero  crossing  has  been  identified  (figure  3(ii)). 

Exact  Interpolation  using  Lagrangian  Polynomials:  If  the  is  no  sign  change 
between  two  support  points  ki  and  then  there  may  be  a  zero  crossing  as 
shown  in  figure  4(iii).  In  that  situation,  a  Lagrangian  polynomial  is  constructed 
using  the  support  points  of  the  current  interval.  The  extreme  point,  cor¬ 
responds  to  a  maximum  or  minumum  with  zero  in  its  first  dervative,  dz/dt  = 
0.  The  assocaited  time  instant,  t* ,  is  found  using  a  Newton’s  Raphson  method. 
The  extremum,  z^^  and  the  initial  value  z®  are  again  checked  for  a  sign  change. 

It  should  be  noted  that  decisions  high  the  hierarchy  are  less  expensive  than 
the  tests  performed  in  subsequent  lower  layers.  In  most  physical  systems,  simple 
root  exclusions  suffices  for  most  of  the  integration  steps.  Typically  99.9  %  of  the 
intervals  have  no  discontinuities  at  all.  Hence,  the  performance  of  root  exclusion 
test  often  determines  the  speed  of  the  entire  algorithm. 
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Fig.  4.  Different  types  of  non-monotonic  event  functions  (i)  no  zero  crossing  (ii)  zero 
crossing  with  a  sign  change  in  the  value  between  the  support  points  (iii)  zero  crossing 
with  no  change  in  sign  of  the  gradients  for  support  points 


3.2  Event  Location 

For  every  event  function  with  a  root  in  the  interval,  the  exact  event  time  needs  to 
be  computed  in  the  event  location  phase.  Since  time  is  the  independent  variable, 
exact  points  of  the  discontinuity  cannot  be  obtained  directly  from  equation  (4) 
and  (5). 


f^{x,y,y,u{t),t)  =  0,te[tn,tn+i]  (4) 

z^^{x*  ,y*  )  ±  ^event  ~  ^  (^) 

In  the  above  equations  (4)  and  (5),  x  are  the  algebraic  variable  values,  y  are 
the  differential  variable  values  and  u(t)  are  known  forcing  functions.  The  star 
indicated  their  value  at  the  event  time,  t*. 

Event  Location  via  implicit  Euler’s  method  and  function  evaluation 
(method  1):  In  order  to  compute  the  exact  event  time,  the  differential  equa¬ 
tions  in  (4)  are  discretised  using  a  first-order  implicit  Euler’s  method.  Discretiza¬ 
tion  renders  algebraic  equations  parameterized  in  the  formerly  independent  time, 
t,  see  Equation  (6).  Together  with  the  event  condition  of  equation  (5),  the  sys¬ 
tem  can  be  solved  for  the  unknown  event  time.  A  small  tolerance.  Seventh  added 
to  the  event  function  in  equation  (6)  ensures  sufficient  zero  penetration.  For  a 
positive  approach  to  a  root,  the  value  of  the  tolerance  is  positive.  The  discretized 
system  is  solved  simultaneously  using  Newton’s  method  to  obtain  the  unknown 
event  time.  In  addition,  we  obtain  the  corresponding  state  variable  values  at  the 
event  time  (x*,  y*,  u*).  The  iteration  converges  rapidly  due  to  extremely  good 
guesses  of  the  initial  values.  This  methodology  also  offers  an  opportunity  for 
controlling  the  sign  and  precision  of  the  event  conditionals,  which  is  paramount 
for  general-purpose  hybrid  simulation. 

+  (6) 

High-order  approximations:  Simulations  using  higher  order  implicit  dis¬ 
cretization  techniques  were  examined  in  order  to  assess  the  precision  of  event 
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location.  The  second-order  semi-implicit  method  leads  to  the  set  of  equation  (7). 
Note  that  the  Jacobian  is  computed  only  once  using  the  values  at  the  beginning 
of  the  interval. 

y*  =y^-h{Riki  +  R2k2)  (7) 

Where, 

ki  =  h[I~haJ{y^)]~^f{y^) 
k2  =  h[I  -  haJ{y°)]-'^f  {y^  +  62/^1) 
a  =  0.435, 62  =  3/4,  ^  1.0358,  R2  =  0.8349 

This  second-order  semi-implicit  approach  requires  more  than  twice  the  num¬ 
ber  of  unknowns  and  more  function  evaluations  as  compared  to  first  order  Euler’s 
method.  In  our  experience  and  in  the  simulation  runs  for  Graver’s  benchmark 
case  studies,  the  event  location  times  obtained  using  first  and  second  order  dis¬ 
cretisation  were  almost  identical.  We  conclude  from  these  experiments  that  first 
order  implicit  Euler’s  method  is  acceptable  for  most  practical  cases. 

Event  Location  via  interpolation  (method  2):  The  second  methodology 
avoids  repeated  function  re-evaluations  altogether  by  interpolation  of  the  all 
trajectories  between  the  discrete  support  points.  This  approach  was  proposed 
by  Barton  [8]  for  differential  algebraic  systems  (DAE).  An  interpolation  in  time 
for  the  state  and  event  functions,  e.g.  can  be  obtained  advantageously  via 

Lagrangian  polynomials,  li(t)  given  in  equation  (8).  It  is  worth  mentioning  that 
the  interpolation  completely  decouples  the  variables  in  the  system  described  by 
equation  (1).  Hence,  the  event  time  can  be  computed  using  a  one-dimensional 
Newton  Raphson  method  to  solve  for  time  t*  only.  With  the  exact  event  time, 
the  computation  of  the  state  variable  value  reduces  to  a  mere  function  evaluation 
of  its  corresponding  interpolation  polynomial,  one  at  a  time. 

^+1  n+l  j. 

n  rzT  (8) 

^event  “  ^  (9) 

Although  method  1  assures  accurate  event  location,  repeated  function  calls  to 
compute  the  derivative  of  the  state  variables  can  be  less  effective.  This  situation 
holds  especially  true  in  large  systems  with  a  small  number  of  event  conditionals. 
For  process  models  loaded  with  involved  physical  property  procedures,  method  2 
for  event  location  is  superior  to  method  1 .  However,  method  1  tends  to  be  more 
accurate  since  it  maintains  variable  dependencies  in  the  local  neighborhood  of 
an  event.  Method  1  may  also  be  superior  in  event-driven  systems  that  involve 
large  numbers  of  event  functions. 

3.3  Step  Completion 

After  a  discontinuity  has  been  located  precisely,  the  consistent  state  transition 
must  be  implemented.  Hence,  step  completion  executes  all  actions  consequent  to 
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the  active  event.  The  actions  may  entail:  (a)  discontinuous  changes  in  the  state 
variables  (b)  a  changeover  to  a  new  set  of  describing  equations  or  (c)  triggering  of 
another  event.  After  step  completion,  control  returns  to  the  regular  drive  routine 
of  the  integrator.  Integration  resumes  involving  a  new  cycle  of  event  handling  in 
subsequent  intervals. 

4  Advanced  Issues  in  Hybrid  Simulation 

4.1  Multiple/Simultaneous  Discontinuities  in  Integration  Step 

Note  that  there  maybe  more  than  one  z-function  that  has  a  root  in  the  interval. 
In  this  situation,  the  exact  location  of  the  event  times,  t*,  allows  to  deduce 
the  active  event.  The  active  event  is  characterized  by  the  earliest  event  time, 
KicUve^  which  is  the  smallest  of  all  event  times  t*.  Therefore,  for  each  candidate 
event,  the  exact  location  of  roots  must  be  found.  Only  the  state  transition  of  the 
active  event  is  executed.  After  firing  the  active  event,  integration  resumes.  The 
handling  of  all  other  events  is  delegated  to  subsequent  integration  steps. 

A  situation  often  omitted  in  hybrid  simulation  deals  with  synchronous  events. 
It  concerns  models  with  two  or  more  events  occurring  with  little  or  no  time  delay. 
In  a  practice,  this  occurs  frequently  when  modeling  multiple  digital  controllers, 
which  samples  at  same  time  instant.  Without  special  treatment,  closeness  of  the 
two  events  could  lead  to  singularity  in  the  integration  method  or  failure  to  detect 
the  events  at  all  [15].  Our  algorithm  handles  such  types  of  events  by  examining 
for  simultaneity  or  near  simultaneity  of  the  events.  The  approach  considers  all 
events  occurring  within  a  time  interval,  Ahsim^  as  simultaneous.  All  instances 
of  simultaneous  events  fire. 

4.2  Step  Size  Selection  after  a  Discontinuity 

Smoothness  of  the  trajectories  is  ensured  by  the  step-size  control  mechanism. 
Therefore,  it  is  important  that  step  acceptance  is  performed  prior  to  event  han¬ 
dling.  After  location  of  a  discontinuity,  the  integration  should  move  away  from 
the  discontinuity  with  large  strides  in  the  interest  of  the  overall  efficiency.  This 
goal  is  antagonistic  to  the  objective  of  small  integration  error.  Three  options 
for  selecting  an  appropriate  step  size,  /inext,  after  location  of  a  discontinuity  are 
discussed. 

One  choice  of  the  new  step  could  be  derived  from  the  step  size,  hevent,  ob¬ 
tained  in  the  event  location  phase.  It  corresponds  to  arbitrary  location  of  an 
event  time  vis-a-vis  the  bounds  of  the  associated  integration  step.  Consequently, 
the  value  of  hevent  could  be  arbitrarily  small.  Since  it  does  not  correlate  with 
the  system  time  constants  at  all,  it  is  an  infeasible  choice  for  a  new  step-size. 

A  better  option  is  to  maintain  the  step  size,  hcurrent,  before  an  event  was 
detected.  This  approach  is  suitable  for  systems  where  the  time  constants  remain 
unaltered  between  states.  If  the  describing  equations  change,  the  new  initial 
value  problem  may  commence  with  multiple  step- size  reduction.  This  behavior 
is  certainly  undesirable. 


398  V.  Bahl  and  A. A.  Linninger 


A  third  approach  deploys  a  fixed  initial  step-size,  hinitiai,  adopted  at  the 
start  time  of  the  integration.  Usually  this  choice  involves  very  small  values  con¬ 
stituting  a  very  conservative  approach.  However,  this  option  gives  users  direct 
control  allowing  them  to  adopt  a  problem-specific  trade-off  between  economy 
and  computational  accuracy.  The  latter  method  was  used  in  the  case  studies  of 
section  5. 


4.3  Selection  of  Tolerances 

State-of-the-art  algorithms  avert  error  propagation  in  numerical  integration  by 
means  of  adaptive  step-size  control.  This  technique  compares  the  differences 
in  numerical  solutions  obtained  by  variable  orders  to  achieve  a  fixed  relative 
accuracy,  srK‘  The  step-size  is  increased  rapidly  when  entering  a  smooth  region. 
If  the  local  truncation  error  leaves  the  desired  tolerance  limits,  the  step-size  is 
reduced  in  the  subsequent  step.  If  the  truncation  error  violates  its  limit,  the  step 
is  rejected  and  re-evaluated  with  a  smaller  step-size.  Event  detection  involves  a 
set  of  new  tolerances  and  adjustable  parameters.  For  optimal  performance  of  the 
entire  procedure,  the  tolerances  of  the  integrator  and  the  event  handling  must 
be  concerted. 

The  tolerance  ±£ event  ensures  sufficient  zero  penetration  as  given  by  equa¬ 
tions  (5)  and  (9).  Its  magnitude  is  chosen  large  enough  to  avoid  discontinuity 
sticking,  while  avoiding  unacceptable  offsets  from  the  exact  event  boundary. 
The  value  of  ±£event  must  be  chosen  larger  than  the  tolerance,  TOLMIN,  which 
bounds  the  residual  equations  in  the  Newton  Raphson  (NR)  Method.  The  second 
tolerance  in  NR  gauges  the  break-oflf  for  function  evaluation,  TOLF,  is  set  to 
2€event‘  is  the  tolerance  for  delineating  simultaneous  events.  The  selected 

tolerance  values  are  shown  in  Table  1. 


Table  1.  Solver  parameters  for  the  algorithm 


Solver  parameter 

Value 

^RK 

£event 

IOSrk 

TOLMIN 

TOLF 

^initial 

lOe/i/c 

5  Application  and  Results 

The  efficiency  of  the  new  algorithm  was  tested  against  benchmark  case  studies. 
Specifics  and  problem  descriptions  can  be  found  in  [6].  A  more  detailed  discus¬ 
sion  of  the  third  benchmark  example  is  presented  in  subsection  5.1.  Section  5.2 
summarizes  the  results  of  the  performance  tests. 
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5.1  A  Rectifier  Circuit  -  Carvers  Example  3 

This  example  concerns  a  rectifier  circuit  as  depicted  in  figure  5(i).  It  consists  of 
two  diodes  D1  and  D2,  two  AC  sources  U1  and  U2,  three  resistances  R1  -  R3 
and  three  inductors  LI  -  L3.  The  balances  for  currents  and  voltages  are  given 
by  equations  (10)  and  (11). 


is  =  H  4- 12 


(10) 


=  Rzih  +  ^2)  +  '^^3(^1  +  ^2) 


(11) 


Fig.  5.  Curcuit  and  state  transition  diagrams  for  Carver’s  example  3 


Table  2.  States  for  the  rectifier  circuit 


State 

State  transitions 

Differential  Equations 

Di  conducting 

(ii  >  0  or  vi  >  V3)  and 
(22  —  0  and  V2  <  V3) 

H  =  (^1  ~  hai)/<^2 

22  =  22  =  0 

Both  Di  and  D2  conducting 

(ii  >  0  or  vi  >  V3)  and 
(22  >  0  and  V2  >  V3) 

2l  =  (os'i^l  +  0.eV2)  +  07^1  +  03^2) 

22  =  {agVi  +  O.10V2)  +  nil*l  +  012*2) 

Only  D2  conducting 

(22  >  0  or  U2  >  V3)  and 
(21  =  0  and  vi  <  113) 

*1  =  *1  =  0 
*2  =  {'^2  —  has)  1(14 

The  constants  are  given  as: 

Ri  =  R2  =  2,i?3  =  10,  Li  =  L2  =  0.04,  L3  =  0.2,  vi  =  -'112  =  100sm(1007rt) 
ai  =  (12, 0.24, 12, 0.24, 13.64,  -11.64,  -50, 0,  -11.64, 13.64, 0,  -50 

The  values  for  h  and  22  differ  depending  upon  three  distinct  states:  (i)  diode 
Di  is  conducting,  (ii)  diode  D2  is  conducting,  and  (iii)  both  Di  and  are 
conducting.  In  effect,  the  system  toggles  between  three  states  as  prescribed  in 
table  2.  The  three  states  correspond  to  four  event  functions  depicted  in  Fig. 
5(ii).  Equations  (12)  describes  the  four  event  functions  that  actually  cause  state 
transitions.  The  event-triggering  state  transitions  can  be  obtained  by  careful  ex¬ 
amination  of  the  expressions  in  the  second  column  of  table  2.  Complex  nested 
event  conditionals  required  in  this  case  study  could  be  expressed  by  means  of 
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a  high-level  modeling  language.  A  description  of  the  hybrid  simulation  environ¬ 
ment  and  its  high-level  language  is  beyond  the  scope  of  this  paper  and  has  been 
described  elsewhere  [16].  Figure  6  (i)  shows  the  current  profiles  obtained  while 
figure  6(ii)  shows  the  voltage  changes  for  vl,  v2  and  v3  in  time. 


Z\=i\,Z2=  h,  Z3=Vi-  Vs,  Z4^V2-  Vs  (12) 


Fig.  6.  (i)  current  profiles  and  (ii)  voltage  variations  in  the  circuit  problem 
5.2  Discussion  of  the  Performance  Results 

Table  4  shows  the  performance  evaluation  for  the  new  algorithm  applied  to 
three  Carvers  benchmark  case  studies.  The  three  examples  clearly  illustrate  the 
strength  of  the  new  root  exclusion  test.  It  detected  all  events  in  strict  time 
order  without  discontinuity  sticking.  It  can  also  be  seen  that  in  most  integration 
intervals  we  have  no  root  at  all  (92.80-99.95  %).  Hence,  the  overall  efficiency  of 
the  algorithm  is  governed  by  the  afficacy  of  the  root  exclusion  test.  The  ratio 
of  non-monotonic  intervals  to  monotonic  intervals  is  typically  small.  This  holds 
especially  true  for  most  of  the  physical  systems  with  discontinuities.  In  Carver’s 
example  2  4.65  %  of  the  intervals  were  non-monotonic.  This  number  is  high  since 
Carver  deployed  periodic  functions  with  atleast  two  non-montonic  intervals  per 
period.  The  algorithm  successfully  eliminated  all  the  non-monotonic  intervals 
without  a  root. 

The  performance  of  the  algorithm  was  also  compared  to  a  Type  I  algorithms, 
see  Table  4.  Our  implementation  of  a  type  I  algorithm  deployed  quadratic  inter¬ 
polation  for  event  location.  The  experiments  further  show  that  Type  I  algorithms 
failed  to  detect  all  the  events  for  Carver’s  example  3.  This  drastic  breakdown 
can  be  explained  by  the  reliance  of  type  I  algorithms  on  function  evaluation 
performed  at  the  bounds  of  each  step.  Hence,  zero-crossings  in  the  middle  of  an 
interval  are  lost.  In  terms  of  function  evaluations,  type  I  and  our  new  method 
is  equivalent.  However,  only  the  new  algorithm  proved  robust  on  all  examples 
with  an  execution  speed  comparable  to  the  fast  type  I  algorithms. 

The  performance  of  the  algorithm  was  also  compared  to  the  interval  arith¬ 
metic  techniques.  The  results  are  shown  in  table  5.  Our  root  exclusion  test  is 
slightly  better  than  the  one  involving  the  interval  arithmetic  techniques.  How¬ 
ever,  there  is  a  substantial  improvement  in  the  rigorous  root  finding  phase  of 
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Table  3.  Performance  Evaluation  of  the  new  algorithm 


Case  Study 

Monotonic 

intervals 

Non-monotonic 

intervals 

Intervals  with 
no  events 

Intervals 

excluded 

Events 

Evaluations 
for  root  excl. 

Carvers  example  1 

39 

3 

92.80 

3 

3 

6 

Carvers  example  2 

164 

8 

95.90 

8 

7 

16 

Carvers  example  3 

22157 

23 

99.95 

23 

10 

46 

Table  4.  Performance  comparison  of  the  new  approach  with  a  Type  I  algorithm 


Example 

Problem  parameters 

Type  I  algorithm 

New  algorithm 

Equations 

Events 

Functions 

Residuals 

Functions 

Residuals 

Carvers  example  1 

2 

1 

739 

589 

1898 

Carvers  example  2 

3 

2 

883 

914 

2930 

Carvers  example  3 

6 

4 

fails 

fails 

1753 

4888 

Table  5.  Comparison  of  the  event  detection  phase  of  the  new  approach  with  interval 
arithmetic 


Event  Detection 

Algebraic  evaluations 
(New  Algorithm) 

Algebraic  evaluations 
(Interval  Arithmetic) 

Root  exclusion  test 

3 

5 

Rigorous  event  detection  phase 

19 

56 

event  detection.  This  enhanced  improvement  in  performance  is  essential  for 
event-driven  hybrid  systems  such  as  simulation  of  digital  regulatory  control  of  a 
physical  process. 


6  Conclusions 


A  fast  and  simple  method  for  hybrid  system  integration  by  means  of  a  multi- 
step  integration  algorithm  with  step-size  control  was  presented.  Our  approach  is 
statistically  motivated  leading  to  a  hierarchical  event  detection  procedure.  The 
improvements  are  due  to  the  exploiting  local  monotonicity  and  smooth  function 
properties  observed  in  algorithms  with  step-size  control.  A  three-layered  hierar¬ 
chy  of  event  exclusion  tests  with  increasing  complexity  safeguards  the  rigor  of  the 
method,  while  upholding  the  performance.  A  cheap  root  exclusion  test  excludes 
roots  of  event  functions  fast.  Even  the  most  expensive  test  in  the  inner  nesting 
level  is  faster  than  existing  approach  based  on  interval  arithmetic.  Phenomena 
not  detected  by  popular  bisection  methods  such  as  same-signed  non-monotonic 
event  functions  with  zero  crossing  are  handled  effectively.  The  method  lays  the 
foundation  for  rapid  simulation  algorithms  as  required  in  new  types  of  dynamic 
optimization  and  optimal  control  problems. 
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Abstract.  In  earlier  work,  we  developed  a  mathematical  hybrid  I/O 
automaton  (HIOA )  modeling  framework,  capable  of  describing  both  dis¬ 
crete  and  continuous  behavior.  This  framework  has  been  used  to  analyze 
examples  of  automated  transportation  systems,  intelligent  vehicle  high¬ 
way  systems,  air  traffic  control  systems,  and  consumer  electronics  appli¬ 
cations.  Here,  we  reconsider  the  basic  definitions  of  the  HIOA  framework, 
in  particular,  the  dual  use  of  external  variables  for  discrete  and  contin¬ 
uous  communication.  We  present  a  new  HIOA  model  that  is  simpler 
than  the  earlier  model,  due  to  a  clearer  separation  between  discrete  and 
continuous  activity. 


1  Introduction 

Recent  years  have  seen  a  rapid  growth  of  interest  in  hybrid  systems — systems 
that  contain  both  discrete  and  continuous  components,  typically  computers  in¬ 
teracting  with  the  physical  world.  Such  systems  are  used  in  many  application 
domains,  including  automated  transportation,  avionics,  automotive  control,  pro¬ 
cess  control,  robotics,  and  consumer  electronics.  Motivated  by  a  desire  to  describe 
and  reason  carefully  about  such  applications,  we  are  continuing  our  efforts  to 
adapt  techniques  from  computer  science  to  the  setting  of  hybrid  systems. 

In  our  previous  work  in  this  area,  we  developed  a  mathematical  hybrid  I/O 
automaton  modeling  framework  [15,16].  This  framework  supports  description 
and  analysis  of  hybrid  systems  using  powerful  methods  of  parallel  composition 
and  levels  of  abstraction.  We  also  proved  sufficient  conditions  for  hybrid  I/O 
automata  to  be  receptive^  which  means  that  they  allow  time  to  advance  to  infinity 
independently  of  the  input  provided  by  the  environment.  We  and  others  have 
used  this  framework  to  analyze  examples  of  automated  transportation  systems 
[18,13,23,22,14,10],  intelligent  vehicle  highway  systems  [6,12],  air  traffic  control 
systems  [11,9],  and  consumer  electronics  systems  [4]. 
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ACI-9876931,  CCR-9909114,  CCR-9804665;  PATH  1784-18454LD. 

**  Supported  by  MURST  project  TOSCA. 

***  Supported  by  Esprit  Project  26270,  Verification  of  Hybrid  Systems  (VHS). 


M.D.  Di  Benedetto,  A.  Sangiovanni-Vincentelli  (Eds.):  HSCC  2001,  LNCS  2034,  pp.  403-417,  2001. 
@  Springer-Verlag  Berlin  Heidelberg  2001 


404  N.  Lynch,  R.  Segala,  and  F.  Vaandrager 


In  this  paper,  we  present  a  new  hybrid  I/O  automaton  model  that  is  con¬ 
siderably  simpler  than  the  earlier  model,  yet  supports  similar  description  and 
analysis  methods  and  similar  receptivity  theorems.  The  main  simplification  is  a 
clearer  separation  between  the  notions  of  discrete  and  continuous  communica¬ 
tion.  We  arrived  at  this  separation  as  a  result  of  reconsidering  the  relationship 
between  the  computer  science  notion  of  shared  variable  communication  and  the 
control  theory  notion  of  continuous  flow  across  component  boundaries. 

Levels  of  abstraction,  compositionality,  and  receptiveness  for  hybrid  systems 
have  also  been  addressed  by  Alur  and  Henzinger  [2,3]  in  their  work  on  reactive 
modules.  However,  reactive  modules  communicate  only  via  shared  variables,  and 
not  via  shared  actions.  In  [3],  a  definition  of  receptiveness  similar  to  the  one  in 
[15,16]  is  proposed,  and  is  shown  to  be  preserved  by  composition.  However,  in  [3], 
no  circular  dependencies  (“feedback  loops”)  are  allowed  among  the  continuous 
variables  of  the  components,  a  restriction  that  greatly  simplifies  the  analysis. 

The  rest  of  this  paper  is  organized  as  follows.  Section  2  defines  notions  that 
are  useful  for  describing  the  behavior  of  hybrid  systems:  trajectories  and  hy¬ 
brid  sequences.  Section  3  contains  the  theory  for  the  hybrid  automaton  (HA) 
model,  which  has  all  of  the  structure  of  the  HIOA  model  except  for  the  division 
of  external  actions  and  variables  into  inputs  and  outputs.  Section  4  introduces 
inputs  and  outputs,  and  presents  the  basic  theory  for  HIOAs.  Section  5  presents 
the  new  theory  of  receptiveness,  including  the  main  theorem.  Theorem  7,  stating 
that  receptiveness  is  preserved  by  composition  under  certain  compatibility  condi¬ 
tions.  Section  6  describes  sufficient  conditions  for  these  compatibility  conditions 
to  hold,  and  in  particular,  describes  Lipschitz  automata. 

2  Describing  Hybrid  Behavior 

In  this  section,  we  give  basic  definitions  that  are  useful  for  describing  discrete  and 
continuous  system  behavior,  including  discrete  and  continuous  state  changes, 
and  discrete  and  continuous  flow  of  information  over  component  boundaries. 
Throughout  this  paper,  we  fix  a  time  axis  T,  which  is  a  compact  subgroup  of 
(R,  -f),  the  real  numbers  with  addition. 

2.1  Static  and  Dynamic  Types 

We  assume  a  universal  set  V  of  variables.  A  variable  represents  either  a  location 
within  the  state  of  a  system  component,  or  a  location  where  information  flows 
from  one  system  component  to  another.  For  each  variable,  we  assume  both  a 
(static)  type^  which  gives  the  set  of  values  it  may  assume,  and  a  dynamic  type, 
which  gives  the  set  of  trajectories  it  may  follow.  Our  motivation  for  introducing 
dynamic  types  is  that  this  allows  us  to  define  input  enabling  for  hybrid  I/O 
automata:  if  v  is  an  input  variable  of  HIOA  A  then,  roughly  speaking,  we  require 
that  A  accepts  each  input  signal  on  t;,  as  long  as  it  respects  the  dynamic  type 
of  V.  Since  we  are  in  a  hybrid  setting  where  discrete  transitions  may  change  the 
state  at  any  time,  elements  of  a  dynamic  type  may  contain  (countably  many) 
“discontinuities”.  Formally,  we  assume  for  each  variable  v\ 
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—  type{v),  the  (static)  type  of  v.  This  is  a  set  of  values. 

-  dtype{v)j  the  dynamic  type  of  v.  This  is  a  set  of  functions  from  left-closed 
intervals  of  T  to  type{v)  that  is  closed  under  the  following  operations: 

1.  (Time  shift)  For  each  /  €  dtype(v)  and  t  G  T,  /  -h  t  €  dtype{v).  Here 
/  -f- 1  is  the  function  given  by  (/  -h  t)  {t')  —  f{t'  —t). 

2.  (Subinterval)  For  each  /  G  dtype{v)  and  each  left-closed  interval  J  C 
dom{f),  f  \  J  G  dtype{v).  Here  /  [  J  is  the  function  obtained  by 
restricting  the  domain  of  /  to  J. 

3.  (Pasting)  For  each  sequence  /o,/i,/25--  -  of  functions  in  dtype{v)  such 
that  (a)  the  domain  of  each  /*,  except  possibly  for  the  last  one,  is  right- 
closed,  (b)  for  each  nonfinal  index  2,  max(dom(/i))  =  min{dom{fi^i)), 
the  function  /  given  by  f{t)  =  fi{t),  where  i  is  the  smallest  index  with 
t  G  dom{fi),  is  in  dtype{v). 


Example  1.  For  any  variable  v,  the  set  C  of  constant  functions  from  a  left-closed 
interval  to  type{v)  is  closed  under  time  shift  and  subintervals.  If  the  dynamic 
type  of  V  is  obtained  by  closing  C  under  the  pasting  operation,  then  v  is  called 
a  discrete  variable,  as  in  [19].  If  we  take  T  =  R  and  type{v)  —  R,  then  other 
examples  of  dynamic  types  can  be  obtained  by  taking  the  pasting  closure  of  the 
set  of  continuous  or  smooth  functions,  the  set  of  integrable  functions,  or  the  set 
of  measurable  locally  essentially  bounded  functions.  The  set  of  all  functions  from 
left-closed  intervals  of  R  to  R  is  also  a  dynamic  type. 

In  practice,  dynamic  types  are  often  defined  via  pasting  closure  of  a  class  of 
continuous  functions.  In  these  cases  the  elements  of  dynamic  types  are  continuous 
from  the  left.  Elsewhere  in  the  literature  on  hybrid  systems  one  often  encounters 
functions  that  are  continuous  from  the  right  (see,  e.g.,  [8]).  To  some  extent, 
the  choice  of  how  to  define  function  values  at  discontinuities  is  arbitrary.  An 
advantage  of  our  choice  is  a  nice  correspondence  between  concatenation  and 
prefix  ordering  of  trajectories  (see  Lemma  2).  In  the  rest  of  this  paper,  when  we 
say  that  the  dynamic  type  of  a  variable  v  equals  5,  we  actually  mean  that  the 
dynamic  type  of  v  is  obtained  by  applying  the  above  closure  operations  to  5. 

2.2  Trajectories 

In  this  subsection,  we  define  the  notion  of  a  trajectory,  define  operations  on 
trajectories,  and  prove  simple  properties  of  trajectories  and  their  operations.  A 
trajectory  is  used  to  model  the  evolution  of  a  collection  of  variables  over  an 
interval  of  time. 


Basic  Definitions.  Let  F  be  a  set  of  variables,  that  is,  a  subset  of  V.  A  valuation 
V  for  F  is  a  function  that  associates  to  each  variable  2;  G  F  a  value  in  type{v). 
We  write  valiV)  for  the  set  of  valuations  for  F.  Let  J  be  a  left-closed  interval 
of  T  with  left  endpoint  equal  to  0.  Then  a  J -trajectory  for  F  is  a  function 
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r  \  J  valiV)^  such  that  for  each  v  ^  V ^  r  v  ^  dtype{v).  Here  r  t;  is  the 
function  with  domain  J  defined  by  (r  i  v){t)  =  T{t){v). 

We  say  that  a  J-trajectory  is  finite  if  J  is  a  finite  interval,  closed  if  J  is  a 
(finite)  closed  interval,  and  full  if  J  =  T-®.  A  trajectory  for  V  is  a  J-trajectory 
for  for  any  J.  We  write  trajs{V)  for  the  set  of  all  trajectories  for  V.  For 
T  a  set  of  trajectories,  finite{T),  closed{T)  and  full{T)  denote  the  subsets  of 
finite,  closed  and  full  trajectories  in  T,  respectively.  A  trajectory  with  domain 
[0, 0]  is  called  a  point  trajectory.  If  v  is  a  valuation  then  p(v)  denotes  the  point 
trajectory  that  maps  0  to  v. 

If  T  is  a  trajectory  then  r.ltime,  the  limit  time  of  r,  is  the  supremum  of 
dom{T).  Similarly,  we  define  r.fval,  the  first  valuation  of  r,  to  be  r(0),  and  if 
r  is  closed,  we  define  r.lval,  the  last  valuation  of  r,  to  be  r (r.ltime).  For  r  a 
trajectory  and  t  e  we  define  r  <t  =  r  f  [0,  t],  r  <  t  =  r  T  [0,t),  and 

T  >  t  =  (t  I"  [t,  oo))  —  t.  Note  that  the  result  of  applying  the  above  operations  is 
always  a  trajectory,  except  when  the  result  is  a  function  with  an  empty  domain. 
By  convention,  r  <  oo  =  r  and  r  <]  oo  =  r. 

Prefiix  Ordering.  Trajectory  r  is  a  prefix  of  trajectory  v,  denoted  by  r  <  l), 
if  r  can  be  obtained  by  restricting  r;  to  a  non-empty,  downward  closed  subset 
of  its  domain.  Formally,  r  <  v  r  =  v  [  Jom(r).  For  T  a  set  of  trajectories 
for  V,  prefiT)  denotes  the  prefix  closure  of  T.  We  say  that  T  is  prefix  closed  if 
T  =  pref{T). 

The  following  lemma  gives  a  simple  domain  theoretic  characterization  of  the 
set  of  trajectories  over  a  given  set  V.  (See  [7]  for  basic  definitions  and  results  on 
complete  partially  ordered  sets,  (cpo’s)). 

Lemma  1.  Let  V  be  a  set  of  variables.  Then  the  set  trajs(V)  of  trajectories 
for  V ,  together  with  the  prefix  ordering  <,  is  an  algebraic  cpo  whose  compact 
elements  are  the  closed  trajectories. 

Concatenation.  The  concatenation  of  two  trajectories  is  obtained  by  taking 
the  union  of  the  first  trajectory  and  the  function  obtained  by  shifting  the  domain 
of  the  second  trajectory  until  the  start  time  agrees  with  the  limit  time  of  the 
first  trajectory;  the  last  valuation  of  the  first  trajectory,  which  may  not  be  the 
same  as  the  first  valuation  of  the  second  trajectory,  is  the  one  that  appears  in 
the  concatenation.  Formally,  let  t,v  be  trajectories,  with  r  closed.  Then  the 
concatenation  is  the  function  given  hy  r'^'v  ^  r  U  (u  [  (0,  oo)  +  r.ltime).  Using 
the  closure  of  dynamic  types  under  time  shift  and  pasting,  it  follows  that  r'^v 
is  a  trajectory.  Observe  that  r  ^  r;  is  finite  (resp.  closed,  full)  iff  v  is  finite  (resp. 
closed,  full).  Observe  also  that  concatenation  is  associative. 

The  following  lemma,  which  is  easy  to  prove,  shows  the  close  connection 
between  concatenation  and  the  prefix  ordering. 

Lemma  2.  Let  r,v  be  trajectories  with  r  closed.  Then  r  <  v  iff  there  exists  a 
trajectory  r'  such  that  r  ^  r' . 


Hybrid  I/O  Automata  Revisited  407 


Note  that  if  r  <  i;,  then  the  trajectory  r'  such  that  u  =  r  ^  r'  is  unique  except 
that  it  has  an  arbitrary  value  for  r'.fval.  Note  also  that  the  “4=”  implication 
would  not  hold  if  the  first  valuation  of  the  second  argument,  rather  than  the  last 
valuation  of  the  first  argument,  were  used  in  the  concatenation. 

Using  a  limit  construction,  we  can  generalize  the  definition  of  concatenation 
for  any  (finite  or  countably  infinite)  number  of  arguments.  Let  to,ti,  r2, . . .  be  a 
(finite  or  infinite)  sequence  of  trajectories,  such  that  is  closed  for  each  nonfinal 
index  i.  Define  trajectories  Tq,  r^,  . . .  by  r-  =  Ti.  We  define 

the  concatenation  tq  ""  Ti  r2 . . .  to  be  limi_).oo  L' •  prove  that 

To  Ti  r2  . . .  is  a  trajectory. 

2.3  Hybrid  Sequences 

In  this  subsection,  we  introduce  the  notion  of  a  hybrid  sequence,  which  is  used 
to  model  a  combination  of  changes  that  occur  instantaneously  and  changes  that 
occur  over  intervals  of  time.  Our  definition  is  parameterized  by  a  set  A  of  actions, 
which  are  used  to  model  instantaneous  changes  and  instantaneous  synchroniza¬ 
tion  with  the  environment,  and  a  set  V  of  variables,  which  are  used  to  model 
changes  over  intervals  and  continuous  interaction.  We  also  define  some  special 
kinds  of  hybrid  sequences  and  operations  on  hybrid  sequences. 

Basic  Definitions.  An  (A,V)-sequence  is  a  finite  or  infinite  alternating  se¬ 
quence  a  =  To  ai  ri  a2  r2  •  •  •,  where  (1)  each  n  is  a  trajectory  in  trajs{V),  (2) 
each  Oi  is  an  action  in  A,  (3)  if  o  is  a  finite  sequence  then  it  ends  with  a  tra¬ 
jectory,  and  (4)  if  is  not  the  last  trajectory  in  a  then  dom{ri)  is  closed.  We 
define  a  hybrid  sequence  to  be  an  (^,  U)-sequence  for  some  A  and  V. 

Since  the  trajectories  in  a  hybrid  sequence  can  be  point  trajectories,  our 
notion  of  hybrid  sequence  allows  a  sequence  of  discrete  actions  to  occur  at  the 
same  real  time,  with  corresponding  changes  of  state. 

If  a  is  a  hybrid  sequence,  with  notation  as  above,  then  we  define  the  first 
valuation  of  a,  a.fval,  to  be  TQ.fval,  and  we  define  the  limit  time  of  a,  adtime, 
to  be  5]].  A  hybrid  sequence  a  is  defined  to  be: 

—  time-bounded  if  a.ltime  is  finite. 

—  admissible  if  a.ltime  =  oo. 

—  closed  if  a  is  a  finite  sequence  and  the  domain  of  its  final  trajectory  is  a 
closed  interval.  In  this  case  we  define  the  last  valuation  of  a,  a.lval,  to  be 
last{a).lval. 

~  Zeno  if  a  is  neither  closed  nor  admissible,  that  is,  if  a  is  time-bounded  and  is 
either  an  infinite  sequence,  or  else  a  finite  sequence  ending  with  a  trajectory 
whose  domain  is  right-open. 

Prefix  Ordering.  We  say  that  {A,  y)-sequence  a;  =  tq  ri . . .  is  a  prefix  of 
{A,  y)-sequence  a'  =  Tq  a[  denoted  by  a  <  a',  if  either  a  =  a',  or  a  is  a 

finite  sequence  ending  in  some  Tk]  n  =  t[,  and  a^+i  =  for  every  z,  0  <  i  <  fc; 
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a^nd  Tk  <  Like  the  set  of  trajectories  over  V,  the  set  of  (A,  y)-sequences  is  a 
cpo. 

Lemma  3.  The  set  of  (A^V)- sequences  together  with  the  prefix  ordering  <  is 
an  algebraic  cpo  with  as  compact  elements  the  set  of  closed  (A^V) -sequences. 


Restriction.  Let  A,  A'  be  sets  of  actions  and  V,  V'  sets  of  variables.  The 
(A',  y')-restriction  of  an  {A,  V^)-sequence  is  obtained  by  projecting  the  trajecto¬ 
ries  on  the  variables  in  removing  the  actions  not  in  A',  and  concatenating 
the  adjacent  trajectories. 

Lemma  4.  Restriction  is  a  continuous  operation  with  respect  to  prefix  ordering. 


Concatenation,  Suppose  a  and  a'  are  (A,  V^)-sequences,  with  a  closed.  Then 
the  concatenation  is  the  (A,  V)-sequence  given  by 

o  a'  =  init[a)  {last[a)  head{a'))  tail{a'). 

(If  cr  is  a  nonempty  sequence  then  head{a)  denotes  the  first  element  of  a  and 
tail  (a)  denotes  a  with  its  first  element  removed;  if  cr  is  finite,  then  last{(T)  denotes 
the  last  element  of  cr  and  init{(T)  denotes  a  with  its  last  element  removed.) 

Lemma  5.  Let  a,  a'  he  [A.^V) -sequences  with  a  closed.  Then  (x  a'  iff  there 
exists  and  (A,  V) -sequence  a"  such  that  a'  =  a  a*'. 

Note  that  if  a  <  a',  then  the  (A,  y)-sequence  a"  such  that  a'  =  a'^a"  is  unique 
except  that  it  has  an  arbitrary  value  in  val(V)  for  a".fval. 

Based  on  Lemma  5  and  Lemma  3,  we  can  extend  concatenation  to  infinitely 
many  (A,  V^)-sequences  as  follows.  Let  Oi,  02, ...  be  an  infinite  sequence  of  closed 
(A,  y)-sequences.  Then  define  the  concatenation  oi 02  •  to  be  lim«_^oo 

where  a[  =  ai  ""  a2  cti. 


3  Hybrid  Automata 

As  a  preliminary  step  toward  defining  hybrid  I/O  automata,  we  first  define  a 
slightly  more  general  hybrid  automaton  model.  Hybrid  automata  classify  actions 
as  external  and  internal,  but  do  not  further  subdivide  the  external  actions  into 
input  and  output  actions.  Likewise,  they  classify  variables  as  external  and  inter¬ 
nal.  The  input/output  distinction  is  added  in  Section  4.  In  addition  to  defining 
hybrid  automata,  we  here  define  an  implementation  relation  between  hybrid 
automata  and  a  composition  operation. 
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3.1  Definition  of  Hybrid  Automata 

A  hybrid  automaton  (HA )  A  ~  (W,  X,G,E,  H,  T>,  T)  consists  of: 


—  A  set  of  external  variables  and  a  set  X  of  internal  variables,  disjoint  from 
each  other.  We  call  a  valuation  x  for  A  a  state,  and  we  refer  to  val{X)  as 
the  set  of  states  of  A.  We  write  V  =  WuX.  Given  a  valuation  v  for  V,  we 
denote  by  state(v)  the  state  v  \  X. 

—  A  nonempty  set  G  C  val{X)  of  start  states. 

—  A  set  E  of  external  actions  and  a  set  H  of  internal  actions,  disjoint  from 
each  other.  We  write  A  —  E\JH  and  let  a,b,.. .  range  over  A. 

—  A  set  C  val{X)  x  A  x  val{X)  of  discrete  transitions.  We  use  x  x' 
as  shorthand  for  (x,  a,  x')  G  V.  We  sometimes  drop  the  subscript,  and  write 
X  A  x',  when  A  should  be  clear  from  the  context. 

~  A  set  T  of  trajectories  for  V.  Given  a  trajectory  r  gT  we  denote  r.fval  \  X 
by  T.f state,  and,  if  r  is  closed,  r.lval  \  X  hy  r.lstate.  We  require  that  the 
following  axioms  hold: 

T1  (Prefix  closure)  For  every  r  gT  and  every  r'  <  r,  r'  G  T. 

T2  (Suffix  closure)  For  every  r  €T  and  every  t  G  dom{T),  r  >  t  G  7”. 

T3  (Concatenation  closure)  Let  ro,Ti,r2,...  be  a  sequence  of  trajectories 
in  T  such  that,  for  each  nonfinal  index  i,  ri  is  closed  and  Ti.lstate  = 
Ti+i.f state.  Then  tq  ^  ti  ^  T2  . . .  e  T. 

Axioms  Tl-3  express  some  natural  closure  properties  on  the  set  of  trajectories 
that  we  need  for  our  results  about  parallel  composition.  In  a  composed  system, 
any  trajectory  of  any  component  may  be  interrupted  at  any  moment  by  a  dis¬ 
crete  transition  of  another  component.  Axiom  T1  ensures  that  the  part  of  the 
trajectory  up  to  the  discrete  transition  is  a  trajectory,  and  axiom  T2  ensures 
the  remainder  is  a  trajectory.  Axiom  T3  is  required  because  the  environment  of 
a  hybrid  automaton,  as  a  result  of  internal  discrete  transitions,  may  change  its 
continuous  dynamics  repeatedly,  and  the  automaton  must  be  able  to  follow  this 
behavior.  Even  without  performing  discrete  transitions  itself,  a  hybrid  automa¬ 
ton  must  be  able  to  follow  this  type  of  behavior  of  its  environment.  In  the  earlier 
definition  of  hybrid  automata  presented  in  [15,16],  we  used  a  special  stuttering 
action  e  in  place  of  axiom  T3;  this  gave  rise  to  technical  complications. 

Another  major  difference  between  our  new  definition  and  the  earlier  one  is 
that  the  external  variables  are  no  longer  considered  to  be  part  of  the  state;  thus, 
for  instance,  the  discrete  transitions  do  not  depend  on  the  values  of  these  vari¬ 
ables.  Analogous  to  the  way  in  which  external  actions  can  be  used  to  model 
synchronization  of  discrete  transitions  of  different  components,  external  vari¬ 
ables  allow  us  to  model  synchronization  of  continuous  activity  ( “flow” )  between 
components.  Because  the  external  actions  and  external  variables  are  not  part  of 
the  state,  we  think  of  them  as  “ephemeral” , 

We  often  denote  the  components  of  a  HA  A  by  W4,  Xj_,  Gj^,  E^,  etc,  and 
the  components  of  a  HA  Ai  by  Wi,  Xi,  Gi,  Ei,  etc.  We  sometimes  omit  these 
subscripts,  where  no  confusion  seems  likely. 
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3.2  Executions  and  Traces 

We  now  define  execution  fragments,  executions,  trace  fragments,  and  traces, 
which  are  used  to  describe  automaton  behavior. 

An  execution  fragment  of  a  HA  A  is  an  {A,  y)-sequence  o  =  tq  ai  ri  a2  T2  •  ■ 
where  (1)  each  is  a  trajectory  in  T,  and  (2)  if  is  not  the  last  trajectory  in 
a  then  Ti.lstate  Ti^i.f state.  An  execution  fragment  records  all  the  instanta¬ 

neous,  discrete  state  changes  that  occur  during  a  specific  evolution  of  a  system, 
as  well  as  the  state  changes  and  external  variable  changes  that  occur  while  time 
advances.  We  write  frags  for  the  set  of  all  execution  fragments  of  A. 

If  a  is  an  execution  fragment,  with  notation  as  above,  then  we  define  the  first 
state  of  a,  a.fstate,  to  be  state(a.fval)^  or  equivalently,  To.fstate.  An  execution 
fragment  a  is  defined  to  be  an  execution  if  a.fstate  is  a  start  state,  that  is,  is  in 
G.  We  write  execs ^  for  the  set  of  all  executions  of  A. 

If  a  is  a  closed  execution  fragment  then  we  define  the  last  state  of  a,  a.lstate, 
to  be  state{a.lval),  or  equivalently,  last  (a).  1st  ate.  A  state  of  A  is  reachable  if  it 
is  the  last  state  of  some  closed  execution  of  A. 

Lemma  6.  Let  a  and  a'  be  execution  fragments  of  A  with  a  closed,  and  such 
that  a.lstate  =  a'.fstate.  Then  a'~"  a'  is  an  execution  fragment  of  A. 

Lemma  7.  Let  a  and  a'  be  execution  fragments  of  A  with  a  closed.  Then  a  <  a' 
iff  there  is  an  execution  fragment  a”  such  that  a'  =  a""  a" . 

The  trace  of  an  execution  fragment  records  the  external  actions  and  the 
evolution  of  external  variables.  Formally,  if  a  is  an  execution  fragment,  then  the 
trace  of  a,  denoted  by  trace{a),  is  the  (E,  W’)-restriction  of  a.  A  trace  fragment 
of  a  hybrid  automaton  A  from  a  state  x  of  .4  is  a  trace  that  arises  from  an 
execution  fragment  of  A  whose  first  state  is  x.  We  write  tracefrags ^(x)  for  the 
set  of  trace  fragments  of  A  from  x.  Also,  we  define  a  trace  of  4.  to  be  a  trace 
fragment  from  an  initial  state,  that  is,  a  trace  that  arises  from  an  execution  of 
A,  and  write  traces  for  the  set  of  traces  of  A. 

Hybrid  automata  Ai  and  A2  are  comparable  if  they  have  the  same  external 
actions  and  variables,  that  is,  if  Wi  =  W2  and  Ei  =  £’2-  If  Ai  and  A2  are 
comparable  then  we  say  that  Ai  implements  A2,  denoted  by  Ai  <  4-2,  if  the 
traces  of  4i  are  included  among  those  of  42,  that  is,  if  traces C  traces 

3.3  Simulation  Relations 

Let  4  and  B  be  comparable  HAs.  A  simulation  from  4  to  is  a  relation  R  C 
val{XjO  X  val{Xis)  satisfying  the  following  conditions,  for  all  states  x^  and  xb 
of  4  and  B,  respectively: 

1.  If  x^  6  0^  then  there  exists  a  state  Xb  ^  Gb  such  that  x^  Rxb- 

2.  If  x^  R  Xb,  xa  -^a  ^  =  trace{p{xA)  a  p(x^)),  then  B  has  a 

closed  execution  fragment  a  with  a.fstate  =  xb,  trace{a)  =  trace{T),  and 
x^  R  a.lstate. 
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3.  If  Xa  R  xb  and  r  is  a  closed  trajectory  of  A  with  x^i  =  r.f state  and  = 
T.lstate,  then  B  has  a  closed  execution  fragment  a  with  a. f state  =  Xb, 
trace{a)  =  trace{r)^  and  x^  R  a.lstate. 


Lemma  8.  Let  A  and  B  be  comparable  HAs,  and  let  R  be  a  simulation  from 
A  to  B.  Let  x^  and  Xb  be  states  of  A  and  B,  respectively,  such  that  xa  Rx.b- 
Then  tracefrags a{^a)  ^  tracefragsjQ{xB)- 


Theorem  1.  Let  A  and  B  be  comparable  HAs,  and  let  R  be  a  simulation  from 
A  to  B.  Then  traces  a  Q  traces  b- 

3.4  Composition 

We  now  introduce  the  operation  of  composition  for  hybrid  automata,  which 
allows  an  automaton  representing  a  complex  system  to  be  constructed  by  com¬ 
posing  automata  representing  individual  system  components.  We  prove  that  the 
composition  operation  respects  our  implementation  relationship  (inclusion  of  sets 
of  traces).  Our  composition  operation  identifies  actions  and  variables  with  the 
same  name  in  different  component  automata.  When  any  component  automaton 
performs  a  step  involving  an  action  a,  so  do  all  component  automata  that  have 
a  in  their  signatures.  Common  variables  are  shared  among  the  components. 

We  define  composition  as  a  partial,  binary  operation  on  hybrid  automata. 
Since  internal  actions  of  an  automaton  Ai  are  intended  to  be  unobservable  by 
any  other  automaton  A2,  we  do  not  allow  Ai  to  be  composed  with  A2  unless 
the  internal  actions  of  are  disjoint  from  the  actions  of  ^2*  Also,  we  require 
disjointness  of  the  internal  variables  of  Ai  and  the  variables  of  ^2-  Formally, 
we  say  that  hybrid  automata  Ai  and  .42  are  compatible  if  for  i  ^  j,  fl  Vj  = 
Hi  n  Aj  —  0.  If  Ai  and  .42  are  compatible  then  their  composition  ^il|.42  is 
defined  to  be  the  structure  A  =  (W,  X,0,E,  H,  V,  T)  where 

-  W  =  WiUW2,  X  =  XiU  X2,  E  -=  EiU  E2,  H  =  HiU  H2. 

-  e  =  {xe  val(X)  I  X  f  Xi  e  6>1  Ax  [  X2  e  6>2}. 

—  For  each  x,  x'  G  val[X)  and  each  a  G  4,  x  x^  iff  for  i  =  1,2,  either  (1) 
a  e  Ai  and  x  \  Xi  -%i  x'  \  Xi,  or  (2)  a  ^  Ai  and  x  [  Xj  =  x'  f  Xj. 

—  7”  C  trajs{  V)  is  given  hy  r  ^  r  \.V\  eTi  A  t4,F2  ^72- 


Proposition  1.  4.i||4.2  is  a  hybrid  automaton. 


Theorem  2.  Suppose  Ai,A2  cb^id  B  are  HAs  with  Ai  <  4.2;  and  suppose  that 
each  of  Ai  and  A2  is  compatible  with  B.  Then  Ai\\B  <  42 1|^- 

In  the  full  version  of  this  paper,  we  define  two  natural  hiding  operations  on 
HAs,  which  hide  external  actions  and  external  variables,  respectively,  and  prove 
that  these  operations  also  respect  the  implementation  preorder. 
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4  Hybrid  I/O  Automata 

In  this  section  we  specialize  the  hybrid  automaton  model  of  Section  3  by  adding 
a  distinction  between  input  and  output. 

4.1  Definition  of  Hybrid  I/O  Automata 

A  hybrid  I/O  automaton  (HIOA)  A  is  a  tuple  {Ti,  U,  Y,  /,  O)  where 

—  71  —  (W,  X,  6>,  E,  Vj  T)  is  a  hybrid  automaton. 

-  U  and  Y  partition  W  into  input  and  output  variables,  respectively.  Variables 
in  Z  =  X  UY  are  called  locally  controlled;  as  before  we  write  V  =  W  UX. 

—  I  and  O  partition  E  into  input  and  output  actions,  respectively.  Actions  in 
L  =  HUO  aie  called  locally  controlled;  as  before  we  write  A  =  EU  H. 

-  The  following  additional  axioms  are  satisfied: 

El  (Input  action  enabling) 

For  all  X  G  val(X)  and  all  a  6  /  there  exists  x'  such  that  x  A  x'. 

E2  (Input  flow  enabling) 

For  all  X  G  val{X)  and  v  G  trajs{U),  there  exists  t  e  T  such  that 
r ./state  =  x,  r  J,  <v,  and  either 

1.  T  lU  =  V,  oi 

2.  there  exist  t  G  dom(T^  and  I  ^  L  such  that  I  is  enabled  from  T[t). 

Input  action  enabling  is  the  input  enabling  condition  of  ordinary  I/O  automata. 
Input  flow  enabling  is  a  new  corresponding  condition  for  continuous  interaction. 
It  says  that  an  HIOA  should  be  able  to  accept  any  continuous  input  flow,  either 
by  letting  time  advance  for  the  entire  duration  of  the  input  flow,  or  by  reacting 
with  a  locally  controlled  action  after  some  part  of  the  input  flow  has  occurred. 

An  execution  of  an  HIOA  A.  is  an  execution  of  Tij^,  Similarly,  a  trace  of  A 
is  a  trace  of  71  a.  Two  HIOAs  Ai  and  A2  are  comparable  if  their  inputs  and 
outputs  coincide,  that  is,  if  E  =  E,  Oi  =-  O2,  Ui  =  C/2,  and  Yi  =  Y2.  If  Ai  and 
A2  are  comparable,  then  Ai  <  A2  is  defined  to  mean  that  the  traces  of  Ai  are 
included  among  those  of  A2:  Ai  <  A2  =  traces  a,  Q  traces  a^^  If  Ai  and  A2  are 
comparable  HIOAs  then  7ii  and  712  are  comparable  and  Ai  <  A2  iff  <  7i2- 
The  definition  of  simulation  for  HIOAs  is  the  same  as  for  HAs,  and  the 
soundness  result  carries  over  immediately  to  the  enriched  setting. 


4.2  Composition 

The  definition  of  composition  for  HIOAs  builds  on  the  corresponding  definition 
for  HAs,  but  also  takes  the  input /output  structure  into  account.  Just  as  in  the 
definition  of  compatibility  for  HAs,  we  do  not  allow  an  HIOA  Ai  to  be  composed 
with  an  HIOA  A2  unless  the  internal  actions  and  variables  of  Ai  are  disjoint 
from  the  actions  and  variables,  respectively,  of  A2.  In  addition,  in  order  that 
the  composition  operation  might  satisfy  nice  properties  (such  as  Theorem  7),  we 
require  that  at  most  one  component  automaton  “controls”  any  given  action  or 
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variable;  that  is,  we  do  not  allow  Ai  and  A2  to  be  composed  unless  the  sets  of 
output  actions  of  Ai  and  A2  are  disjoint  and  the  sets  of  output  variables  of  Ai 
and  A2  are  disjoint. 

If  Ai  and  A2  are  compatible  then  their  composition  ^i||.4.2  is  defined  to  be 
the  tuple  A  =  {U,U,Y,I,0)  where  U  =  'hi\\H2,  U  =  {UiU  U2)  -  (Vi  U  72), 
7  =  7i  U  72,  /  =  (/i  U  I2)  -  (Oi  U  O2),  and  O  =  Oi  U  O2. 

The  definition  of  compatibility  given  above  is  not  quite  strong  enough  to 
imply  that  the  composition  of  two  HIOAs  is  actually  an  HIOA.  Thus,  we  de¬ 
fine  a  stronger  notion  and  say  that  compatible  HIOAs  A\  and  A2  are  strongly 
compatible  if  .Ai||^2  satisfies  axiom  E2.  Strong  compatibility  implies  that  the 
reaction  of  the  composed  automaton  to  any  input  flow  v  must  be  the  result  of  a 
deliberate  reaction  by  either  Ai  or  ^42-  That  is,  either  both  Ai  and  A2  accept  v 
in  its  entirety,  or  one  of  the  two  reacts  with  a  locally  controlled  action.  No  “time 
deadlock”  is  allowed  due  to  incompatible  reactions  of  Ai  and  A2. 

Proposition  2.  The  composition  of  two  strongly  compatible  HIOAs  is  an  HIOA, 


Theorem  3.  Suppose  >4i,^2  B  are  HIOAs  with  Ai  <  A2,  each  of  Ai 
and  A2  is  strongly  compatible  with  B.  Then  <  ^2||^- 

5  Receptive  Hybrid  I/O  Automata 

In  this  section  we  adapt  the  notion  of  receptiveness  [20]  to  our  new  framework. 
Informally  speaking,  a  system  is  receptive  provided  that  it  admits  a  strategy  for 
resolving  its  nondeterministic  choices  that  never  generates  infinitely  many  locally 
controlled  actions  in  finite  time.  An  important  consequence  of  this  definition  is 
that  a  receptive  HIOA  has  some  response  defined  for  any  sequence  of  discrete 
and  continuous  input.  We  show  that  receptiveness  is  closed  under  composition. 
Because  of  the  improvements  in  our  new  model,  the  treatment  of  receptiveness 
in  this  paper  is  simpler  than  that  in  [20];  however,  we  only  address  admissibility 
here,  and  not  general  liveness  properties  as  in  [20]. 

An  execution  fragment  of  an  HIOA  is  locally- Zeno  if  it  is  Zeno  and  contains 
infinitely  many  locally  controlled  actions.  An  HIOA  A  is  locally-Zeno  if  it  has 
at  least  one  locally-Zeno  execution  fragment.  In  the  rest  of  the  paper  we  will  be 
interested  mainly  in  non-locally-Zeno  HIOAs,  that  is,  HIOAs  that  are  not  locally- 
Zeno.  We  use  non-locally-Zeno  HIOAs  as  the  basis  for  defining  receptiveness. 

Theorem  4.  Let  Ai,  A2  be  strongly  compatible  non-locally-Zeno  HIOAs.  Then 
11*42  is  also  non-locally-Zeno. 


Theorem  5.  Let  A  be  a  non-locally-Zeno  HIOA.  Then,  for  each  (7,  U)-sequence 
j3  and  each  state  x,  there  is  an  execution  fragment  a  of  A  such  that  (1)  a.  f state  = 
X,  (2)a\{I,U)  =  p. 
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The  property  stated  in  Theorem  5  is  known  in  the  literature  as  I/O  feasi¬ 
bility  [17];  it  implies  that  any  finite  execution  can  be  extended  to  an  admissible 
execution,  no  matter  what  the  environment  does. 

A  strategy  for  an  HIOA  A  is  an  HIOA  A’  that  differs  from  A  only  in  that  V'  C 
V  and  7”'  C  7~.  A  strategy  A'  for  an  HIOA  A  can  be  viewed  as  a  nondeterministic 
memoryless  strategy  in  the  sense  of  [5,20]  that  chooses  some  of  the  evolutions 
that  are  possible  from  each  of  the  states  of  A.  The  fact  that  the  states  of  A  and 
A!  are  the  same  ensures  that  A'  chooses  evolutions  for  every  state  x  of  A. 

We  say  that  an  HIOA  is  receptive  if  it  has  a  non- locally- Zeno  strategy. 

Theorem  6.  A  receptive  HIOA  is  I/O  feasible. 

Theorem  7.  Let  Ai  and  A2  be  two  compatible  receptive  HIOAs  with  two 
strongly  compatible  non-locally-Zeno  strategies  Ai  and  A2,  respectively.  Then 
^i||v42  is  a  receptive  HIOA  with  non-locally-Zeno  strategy 

6  Sufficient  Conditions  for  Strong  Compatibility 

In  order  to  apply  Theorem  7,  one  has  to  establish  that  two  strategies  are  strongly 
compatible.  This  is  difficult  in  general  since  it  requires  checking  compatibility 
between  the  continuous  dynamics  of  two  systems.  However,  for  certain  restricted 
classes  of  HIOAs,  strong  compatibility  follows  directly  from  compatibility. 

6.1  HIOAs  with  Restrictions  on  Input  Variables 

Our  first  example  is  the  class  of  HIOAs  without  input  variables.  It  is  routine  to 
verify  that  two  HIOAs  without  input  variables  are  strongly  compatible  iff  they 
are  compatible.  From  the  perspective  of  classical  control  theory  a  system  without 
input  variables  is  uninteresting  because  it  cannot  be  controlled;  in  a  hybrid 
setting,  however,  a  system  without  input  variables  can  still  interact  with  its 
environment  via  discrete  input  actions.  Linear  hybrid  automata  [1],  for  instance, 
have  no  input  variables. 

Another  example  is  the  class  of  autistic  HIOAs— those  for  which  the  values 
of  output  variables  do  not  depend  on  the  values  of  input  variables.  Formally, 
an  HIOA  A  is  called  autistic  if  for  all  r  e  T  and  all  v  G  trajs{U)  such  that 
dom{r)  =  dom{v)  there  exists  r'  €  T  such  that  r'  |  C/  =  u  and  r'  j.  F  =  r  |  T. 

6.2  Lipschitz  HIOAs 

In  this  section,  we  define  Lipschitz  HIOAs,  based  on  systems  of  differential  equa¬ 
tions  using  Lipschitz  functions.  We  give  examples  of  conditions  on  classes  of 
Lipschitz  HIOAs  that  imply  strong  compatibility.  The  ideas  are  derived  from 
methods  in  the  literature  on  control  theory  [21].  In  control  theory,  continuous 
system  behavior  is  typically  defined  using  differential  equations  of  the  form: 

D  =  f  ^  ~  f  (^5 
\y  =  9{x) 
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where  u,  y,  and  x  are  the  vectors  of  input,  output,  and  state  variables,  respec¬ 
tively,  together  with  a  starting  condition  of  the  form  x{0)  =  xq. 

To  ensure  that  the  system’s  behavior  is  defined,  the  differential  equations 
must  admit  a  solution  for  each  possible  starting  condition.  The  following  theorem 
from  calculus  gives  sufficient  conditions  for  a  solution  to  exist. 

Theorem  8  (Local  existence).  If  f  is  globally  Lipschitz  and  u  is  then  for 
each  starting  condition  a[:(0)  =  xq  there  is  a  unique  solution  to  the  equations  of 
Dj  defined  on  a  maximal  neighborhood  of  0,  such  that  x(0)  =  xq. 

Observe  that,  since  the  set  of  globally  Lipschitz  functions  is  closed  under  com¬ 
position,  the  local  existence  theorem  is  valid  also  when  the  variables  w  are  the 
result  of  a  globally  Lipschitz  function  applied  to  a  (7^  function. 

Suppose  two  interacting  systems  are  described  by  sets  of  equations  Di  and 
D2  of  the  form  given  above.  Then  their  combined  behavior  can  be  described  by 
the  union  of  the  sets  of  equations  Di  and  1)2 ■  It  is  easy  to  show  that,  if  the 
functions  occurring  in  Di  and  D2  are  globally  Lipschitz,  and  Di  and  D2  do 
not  have  any  common  output  and  state  variables,  then  the  union  of  these  two 
sets  of  equations  is  expressible  in  the  same  form  with  functions  that  are  globally 
Lipschitz.  Thus,  in  this  case  no  additional  machinery  is  needed  to  prove  that 
the  behavior  of  the  interacting  systems  is  well  defined.  We  define  a  set  D  of 
equations  to  be  Lipschitz  if  functions  /  and  g  are  globally  Lipschitz. 

To  extend  the  above  ideas  to  the  hybrid  case  we  define  the  notion  of  a  Lips¬ 
chitz  HIOA.  An  HIOA  A  is  Lipschitz  if  there  is  a  subset  M  of  its  state  variables 
(we  call  these  the  mode  variables)  such  that: 

LI  The  dynamic  type  of  each  variable  in  M  is  piecewise  constant. 

L2  The  dynamic  type  of  each  variable  not  in  Af  is  a  subset  of  the  set  of  real¬ 
valued  functions  defined  on  left-closed  intervals  of  the  reals  that  can  be 
expressed  in  the  form  h{c{’))  where  h  is  a  globally  Lipschitz  function  and  c 
is  a  function,  closed  under  pasting. 

L3  The  values  of  the  M  variables  are  constant  in  each  trajectory  of  T - 
L4  For  each  valuation  m  of  M  there  is  a  Lipschitz  system  of  equations 

with  input  variables  U,  output  variables  T,  and  state  variables  X  -  M  such 
that  the  following  holds:  If  trajectory  r  of  T  starts  from  a  state  x  with 
X  [  M  =  m,  then  r  [  F  -  M  is  expressible  as  the  concatenation  of  countably 
many  trajectories  ro,ri, . . .,  where  each  is  a  solution  to  Dm- 

Define  a  Lipschitz  HIOA  to  be  input  bounded  if  for  each  input  variable  u 
there  exists  a  positive  real  value  B  such  that  every  function  in  the  dynamic  type 
of  u  has  range  in 

Lemma  9.  Compatible  input-bounded  Lipschitz  HIOAs  are  strongly  compatible. 


Theorem  9.  The  composition  of  two  compatible  input-bounded  Lipschitz  HIOAs 
is  a  Lipschitz  HIOA. 
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Theorem  10.  Let  Ai  and  A.2  be  compatible  receptive  HIOAs  with  non-locally- 
Zeno,  input-bounded,  Lipschitz  strategies.  Then  Ai\\A2  is  a  receptive  HIOA  with 
a  non-locally-Zeno  input-bounded  Lipschitz  strategy. 

Theorem  11.  The  composition  of  two  compatible  receptive  input-bounded  Lip¬ 
schitz  HIOAs  is  a  receptive  input-bounded  Lipschitz  HIOA. 

The  conclusion  that  we  derive  from  Theorem  11  is  that  compatibility  implies 
strong  compatibility  if  we  describe  the  continuous  behaviors  of  HIOAs  by  means 
of  differential  equations  of  the  form  of  D  with  functions  /  and  g  globally  Lip¬ 
schitz.  In  general,  any  choice  of  conditions  on  /,  g,  and  u  that  guarantees  local 
existence  of  unique  solutions,  continuity  of  solutions,  and  that  is  preserved  by 
interaction  between  systems,  can  be  used  to  define  a  class  of  automata  for  which 
strong  compatibility  follows  from  compatibility. 


References 

1.  R.  Alur,  C.  Courcoubetis,  N.  Halbwachs,  T.A.  Henzinger,  P.-H.  Ho,  X.  Nicollin, 
A.  Olivero,  J.Sifakis,  and  S.  Yovine.  The  algorithmic  analysis  of  hybrid  systems. 
Theoretical  Computer  Science,  138:3-34,  1995. 

2.  R.  Alur  and  T.A.  Henzinger.  Reactive  modules.  Proc.  LICS’96,  pp.  207-218,  1996. 

3.  R.  Alur  and  T.A.  Henzinger.  Modularity  for  timed  and  hybrid  systems.  In  Proc. 
of  CONCUR’97,  LNCS  1243,  pp.  74-88,  1997. 

4.  D.J.B.  Bosscher,  I.  Polak,  and  F.W.  Vaandrager.  Verification  of  an  audio  control 
protocol.  In  Proc.  of  FTRTFT94,  LNCS  863,  pp.  170-192,  1994. 

5.  D.  Dill.  Trace  Theory  for  Automatic  Hierarchical  Verification  of  Speed- Independent 
Circuits.  ACM  Distinguished  Dissertations.  MIT  Press,  1988. 

6.  E.  Dolginova  and  N.A.  Lynch.  Safety  verification  for  automated  platoon  maneu¬ 
vers:  A  case  study.  Proc.  of  HART97,  LNCS  1201,  pp.  154-170,  1997. 

7.  C.A.  Gunter.  Semantics  of  Programming  Languages:  Structures  and  Techniques. 
MIT  Press,  Cambridge,  Massachusetts,  1992. 

8.  A.  Kapur,  T.A.  Henzinger,  Z.  Manna,  and  A.  Pnueli.  Proving  safety  properties  of 
hybrid  systems.  In  Proc.  of  FTRTFT’94,  LNCS  863,  pp.  431-454,  1994. 

9.  C.  Livadas,  J.  Lygeros,  and  N.A.  Lynch.  High-level  modelling  and  analysis  of 
TCAS.  In  Proc.  of  RTSS ’99,  1999. 

10.  C.  Livadas  and  N.A.  Lynch.  Formal  verification  of  safety-critical  hybrid  systems. 
In  Proc.  of  HSCC’98,  LNCS  1386,  pp.  253-272,  1998. 

11-  J.  Lygeros  and  N.A.  Lynch.  On  the  formal  verification  of  the  TCAS  conflict 
resolution  algorithms.  In  Proc.  of  36th  IEEE  Conference  on  Decision  and  Control, 
pp.  1829-1834,  1997.  Extended  abstract. 

12.  J.  Lygeros  and  N.A.  Lynch.  Strings  of  vehicles:  Modeling  and  safety  conditions. 
In  Proc.  of  HSCC’98,  LNCS  1386,  pp.  273-288,  1998. 

13.  N.A.  Lynch.  Modelling  and  verification  of  automated  transit  systems,  using  timed 
automata,  invariants  and  simulations.  In  Hybrid  Systems  HI,  LNCS  1066,  1996. 

14.  N.A.  Lynch.  A  three-level  analysis  of  a  simple  acceleration  maneuver,  with  uncer¬ 
tainties.  Proc.  of  AMAST  Workshop  on  Real-Time  Systems,  pp.  1-22,  1996. 

15.  N.A.  Lynch,  R.  Segala,  F.W.  Vaandrager,  and  H.B.  Weinberg.  Hybrid  I/O  au¬ 
tomata.  In  Hybrid  Systems  III,  LNCS  1066,  pp.  496-510,  1996. 


Hybrid  I/O  Automata  Revisited  417 


16.  N.A.  Lynch,  R.  Segala,  F.W.  Vaandrager,  and  H.B.  Weinberg.  Hybrid  I/O  au¬ 
tomata.  Report  CSI-R9907,  Computing  Science  Institute,  Univ.  of  Nijmegen,  1999. 

17.  N.A.  Lynch  and  F.W.  Vaandrager.  Action  transducers  and  timed  automata.  For¬ 
mal  Aspects  of  Computing,  8(5):499-538,  1996. 

18.  N.A.  Lynch  and  H.B.  Weinberg.  Proving  correctness  of  a  vehicle  maneuver:  Decel¬ 
eration.  Proc.  European  Workshop  on  Real-Time  and  Hybrid  Systems,  1995. 

19.  O.  Maler,  Z.  Manna,  and  A.  Pnueli.  From  timed  to  hybrid  systems.  In  Proc.  REX 
Workshop  on  Real-Time:  Theory  in  Practice,  LNCS  600,  pp.  447-484,  1992. 

20.  R.  Segala,  R.  Gawlick,  J.F.  S0gaard-Andersen,  and  N.A.  Lynch.  Liveness  in  timed 
and  untimed  systems.  Information  and  Computation,  141(2):  119-171,  March  1998, 

21.  E.D.  Sontag.  Mathematical  Control  Theory  —  Deterministic  Finite  Dimensional 
Systems,  volume  6  of  Texts  in  Applied  Mathematics.  Springer- Verlag,  1990. 

22.  H.B.  Weinberg  and  N.A.  Lynch.  Correctness  of  vehicle  control  systems:  A  case 
study.  In  Proc.  RTSS’96,  pp.  62-72,  1996. 

23.  H.B.  Weinberg,  N.A.  Lynch,  and  N.  Delisle.  Verification  of  automated  vehicle 
protection  systems.  In  Hybrid  Systems  III,  LNCS  1066,  pp.  101-113,1996. 


Validating  a  Hamilton- Jacobi  Approximation  to 
Hybrid  System  Reachable  Sets* 


Ian  Mitchell^**,  Alexandre  M.  Bayen^,  and  Claire  J.  Tomlin^ 


^  Scientific  Computing  and  Computational  Mathematics  Program, 
Gates  2B,  Stanford  University,  Stanford,  CA,  94305,  USA 
mitchellQsccm . Stanford . edu 
^  Department  of  Aeronautics  and  Astronautics, 

Durand  250,  Stanford  University,  Stanford,  CA,  94305,  USA 
{bayen ,  toinlin}@st  anf  ord .  edu 


Abstract.  We  develop  a  general  framework  for  solving  the  hybrid  sys¬ 
tem  reachability  problem,  and  indicate  how  several  published  techniques 
fit  into  this  framework.  The  key  unresolved  need  of  any  hybrid  system 
reachability  algorithm  is  the  computation  of  continuous  reachable  sets; 
consequently,  we  present  new  results  on  techniques  for  calculating  nu¬ 
merical  approximations  of  such  sets  evolving  under  general  nonlinear 
dynamics  with  inputs.  Our  tool  is  based  on  a  local  level  set  procedure 
for  boundary  propagation  in  continuous  state  space,  and  has  been  im¬ 
plemented  using  numerical  schemes  of  varying  orders  of  accuracy.  We 
demonstrate  the  numerical  convergence  of  these  schemes  to  the  viscosity 
solution  of  the  Hamilton- Jacobi  equation,  which  was  shown  in  earlier 
work  to  be  the  exact  representation  of  the  boundary  of  the  reachable 
set.  We  then  describe  and  solve  a  new  benchmark  example  in  nonlinear 
hybrid  systems:  an  auto- lander  for  a  commercial  aircraft  in  which  the 
switching  logic  and  continuous  control  laws  are  designed  to  maximize 
the  safe  operating  region  across  the  hybrid  state  space. 


1  Introduction 

The  focus  of  this  paper  is  the  development  and  numerical  validation  of  a  compu¬ 
tational  tool  to  perform  as  exact  as  possible  reachability  computation  and  con¬ 
troller  synthesis  for  nonlinear  hybrid  systems.  As  such,  we  draw  on  our  previous 
work  in  which  we  characterized  the  boundary  of  the  reachable  set  of  a  hybrid 
system  as  the  zero  level  set  of  the  viscosity  solution  of  a  particular  Hamilton- 
Jacobi  equation  [1],  and  in  which  we  showed  that  it  was  feasible  to  compute  this 
zero  level  set  using  so-called  “level  set  methods”  [2].  The  current  paper  reflects 
our  progress  in  the  development  of  a  general  purpose  tool  for  this  reachable 
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set  computation — the  core  of  which  is  a  new  variant  of  a  “local  level  set”  al¬ 
gorithm  that  more  efficiently  computes  a  more  accurate  representation  of  the 
reachable  set  boundary.  In  addition,  we  demonstrate  the  numerical  convergence 
of  our  computation  by  analyzing  the  results  as  the  continuous  state  space  grid 
is  made  finer,  a  standard  method  of  validation  for  scientific  computing  codes.  In 
this  way,  we  show  that  high  accuracy  can  be  achieved  at  the  cost  of  increased 
computational  time  and  space.  We  illustrate  our  tool  on  a  single  mode  aircraft 
conflict  resolution  example  [2,3],  as  well  as  on  a  new  benchmark  example  of  a 
six  mode  commercial  aircraft  auto-lander,  which  exhibits  nondeterminism  and 
cycles  in  its  discrete  behavior. 

Our  motivation  for  this  project  stems  from  the  belief  that  for  many  applica¬ 
tions  of  hybrid  systems,  it  is  important  to  be  able  to  accurately  represent  the 
reachable  set.  We  have  dealt  primarily  in  the  safety  verification  of  avionic  sys¬ 
tems,  where  accurate  representation  of  the  safe  region  of  operation  translates 
into  the  ability  to  operate  the  system  closer  to  the  boundaries  of  that  region,  at 
a  higher  performance  level  than  previously  allowed.  For  very  high  dimensional 
state  spaces,  additional  logic  (such  as  projection  operators)  or  new  techniques 
(such  as  convex  overapproximations)  will  be  needed;  however,  our  results  in  this 
paper  show  that  it  is  feasible  to  do  exacting  computation  for  hybrid  systems 
with  nonlinear  continuous  dynamics  in  three  continuous  state  dimensions  and 
six  discrete  modes,  and  we  believe  it  will  be  feasible  to  extend  this  up  to  five 
continuous  dimensions  and  large  numbers  of  discrete  modes. 

2  Reachability  for  Hybrid  Systems 

Assuming  that  tools  for  discrete  and  continuous  reachability  are  available — we 
postpone  to  subsequent  sections  the  problems  of  creating  such  tools — computing 
reachable  sets  for  hybrid  systems  requires  keeping  track  of  the  interplay  between 
these  discrete  and  continuous  tools.  In  this  section  we  summarize  the  general 
framework  for  handling  this  interaction  (following  [1]),  and  we  show  how  various 
hybrid  system  reachability  algorithms  described  in  the  literature  fit  into  this 
framework. 

Fundamentally,  reachability  analysis  in  discrete,  continuous  or  hybrid  sys¬ 
tems  seeks  to  partition  states  into  two  categories:  those  that  are  reachable  from 
the  initial  conditions,  and  those  that  are  not.  We  will  label  these  two  sets  of 
states  G  and  E  =  respectively. 

Any  inputs  to  the  hybrid  automata  are  assumed  to  lie  in  bounded  sets  and  to 
have  the  goal  of  locally  maximizing  or  minimizing  the  reachable  set:  at  each  iter¬ 
ation,  the  reachability  algorithm  chooses  values  for  inputs  that  maximize  the 
size  of  G  and  values  for  inputs  that  minimize  the  size  of  G  (and  hence  maxi¬ 
mize  the  size  of  E).  Any  nondeterminism  in  the  transition  relation  is  also  utilized 
to  consistently  maximize  or  minimize  G,  depending  on  the  goal  of  the  reacha¬ 
bility  computation.  For  hybrid  automata,  the  discrete  inputs  a  and  continuous 
inputs  i'  can  be  assigned  to  the  two  categories  and  =  (^^e,^^e) 

according  to  whether  they  seek  to  maximize  or  minimize  G. 
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Fig.  1.  Iterative  Reachability  Algorithm:  Showing  detail  of  iteration  for  discrete  mode 
k  at  iteration  i. 


The  reachability  computation  follows  an  iterative,  two  stage  algorithm  shown 
graphically  in  Figure  T  The  outer  iteration  computes  reachability  over  the  dis¬ 
crete  switches,  producing  iterates  and  at  iteration  i  =  1, 2, . . . .  The  inner 
iteration  runs  a  separate  continuous  reachability  problem  in  each  of  the  discrete 
modes  y  =  1, 2, . . .  RT  to  compute  the  estimates  Gj  and  EL  We  define  the  “switch” 
sets 

-  G^  contains  all  states  in  mode  j  from  which  a  discrete  transition  to  a  state 
in  Gi_i  (typically  a  state  in  another  mode)  can  be  forced  to  occur  through 
the  application  of  a  discrete  input  <tg;  these  states  will  be  defined  by  the 
invariant  of  mode  j  and  the  guards  of  the  transitions  from  mode  j. 

~  Ej  contains  all  states  from  which  a  discrete  transition  to  a  state  in  Ei_i  can 
be  forced  to  occur  through  the  application  of  a  discrete  input  cte;  these  states 
are  also  defined  by  the  invariant  of  mode  j  and  the  guards  of  transitions  from 
mode  j. 

Then  the  goal  of  the  continuous  reachability  tool  is  to  identify  the  “flow”  sets 

-  Gj  (t)  contains  states  from  which  for  all  z^e  there  exists  i^g  that  will  force  the 
^suiting  trajectory  to  flow  into  U  G^  within  time  t. 

-  Ej(t)  contains  states  from  which  there  exists  z/e  that  for  all  i/q  will  force 
the  resuming  trajectories  to  flow  into  E^  within  time  t  or  to  stay  outside  of 

U  G^  for  at  least  time  t. 

Note  that  in  some  problems  the  order  of  the  existential  and  universal  quantifiers 
in  the  definition  above  must  be  reversed.  Given  these  sets, 
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where  Cj  is  the  set  of  initial  conditions  of  the  reachability  problem  and  EJ  = 
Simple  modifications  of  this  algorithm  suffice  to  solve  finite  time  reacha¬ 
bility  problems. 

The  procedure  described  above,  developed  in  [1,3],  was  motivated  by  the 
work  of  [4,5]  for  reachability  computation  and  controller  synthesis  on  timed 
automata,  and  that  of  [6]  for  controller  synthesis  on  linear  hybrid  automata.  In 
that  development  the  reachability  problem’s  objective  was  to  determine  E— the 
largest  controllable  invariant  subset  of  the  state  space — by  computing  the  set 
of  states  G  which  were  reachable  in  backwards  time  from  the  set  of  predefined 
unsafe  states.  In  terms  of  the  definitions  above,  control  inputs  from  this  problem 
lie  in  and  disturbance  inputs  in  ^g-  For  safety,  any  model  nondeterminism 
would  be  used  to  maximize  the  unsafe  set  G. 

Other  hybrid  system  reachability  algorithms  fall  within  this  framework;  the 
differences  lie  in  their  discrete  and  continuous  reachability  solvers  and  the  types 
of  initial  conditions,  inputs,  invariants  and  guards  that  they  admit.  Most  are 
described  as  running  forwards  in  time  from  a  set  of  safe  initial  conditions,  in 
which  case  G  is  computed  as  the  smallest  controllable  invariant  set.  For  exam¬ 
ple,  in  [7,8]  reachability  is  run  with  as  the  controlled  inputs  and  as  the 
disturbance  inputs  with  the  resulting  safe  set  as  G.  The  CheckMate  tool  [9]  deals 
with  threshold  event-driven  hybrid  systems — meaning  that  switches  are  both 
enabled  and  forced  only  at  hyperplanes  in  the  continuous  state  space — so  there 
is  no  equivalent  to  cte  and  thus  E]  =  0.  Because  VeriSHIFTs  algorithm  [10] 
is  designed  for  bounded  time,  decidability  can  be  proven  for  certain  hybrid  au¬ 
tomata.  If  we  are  willing  to  forgo  decidability  then  its  extension  to  infinite  time 
is  straightforward  and  produces  a  reachability  procedure  similar  in  expressive 
capacity  to  CheckMate,  albeit  for  different  continuous  representations. 

3  Continuous  Reachability  with  Level  Sets 

While  practical  algorithms  for  computing  discrete  reachability  over  many  thou¬ 
sands  of  states  have  been  designed  and  implemented,  determination  of  continu¬ 
ous  reachability  for  even  low  dimensional  systems  is  still  an  open  problem.  The 
continuous  portion  of  a  hybrid  reachability  problem  requires  methods  of  per¬ 
forming  four  key  operations  on  sets:  unions,  intersections,  tests  of  equality,  and 
evolution  according  to  the  discrete  mode’s  continuous  flow  field.  The  choice  of 
representation  for  sets  dictates  the  complexity  and  accuracy  of  these  operations; 
consequently,  continuous  reachability  algorithms  can  be  classified  according  to 
how  they  represent  sets. 

Polygonal  representations  have  proven  the  most  popular.  The  tool  d/dt  [7, 
11]  tracks  the  motion  of  convex  polyhedra  under  linear  flow,  collecting  the  non- 
convex  union  of  this  result  into  “orthogonal  polyhedra”  [12].  The  developers 
of  CheckMate  describe  optimization  based  methods  of  tracking  convex  poly¬ 
hedra  under  general  flows,  including  specializations  for  the  affine  case  [13,14]. 
Projectagons  [15]  is  the  term  used  to  describe  the  idea  of  storing  nonconvex 
high  dimensional  polyhedra  as  the  intersection  of  two  dimensional  projections, 


422  I.  Mitchell,  A.M.  Bayen,  and  C.J.  Tomlin 


which  are  evolved  under  affine  over  approximations  of  general  flows  using  linear 
programming.  VeriSHIFT  [10]  uses  ellipsoidal  representation  of  reach  sets  for 
linear  flows  with  linear  input;  it  implements  techniques  developed  in  [16]. 

3.1  The  Hamilton- Jacobi  Partial  Differential  Equation 

For  our  representation  scheme,  we  characterize  the  set  being  tracked  implicitly 
by  defining  a  “level  set  function”  J{x,t)  throughout  the  continuous  state  space 
which  is  negative  inside  the  set,  zero  on  its  boundary,  and  positive  outside,  and 
which  encodes  the  initial  data  in  J(x,0).  The  intersection  of  two  such  sets  is 
simply  the  maximum  of  their  level  set  functions  at  each  point  in  state  space, 
and  the  union  is  the  minimum;  a  variety  of  easily  implemented  equality  tests  are 
possible.  Evolution  of  a  level  set  under  a  nonlinear  flow  field  is  governed  by  the 
Hamilton- Jacobi  (HJ)  partial  differential  equation  (PDE)  (see,  for  example,  [2]) 

dJ(x,t)  .  ^ 

^1^ /(^)  ^min?  ^max)  (1) 

^min  i^max 

=  H{x,VJ{x,t)).  (2) 

where  r^niin  those  continuous  inputs  trying  to  minimize  the  size  of  the  set 
being  tracked,  and  are  those  inputs  trying  to  maximize  its  size.  The  order 
of  the  optimization  must  be  chosen  appropriately  for  the  situation.  The  implicit 
representation  has  a  number  of  advantages  when  compared  with  the  explicit 
representations  that  other  researchers  are  pursuing,  including  a  conceptually 
simple  representation  of  very  general  sets  and  a  size  which  is  independent  of 
the  complexity  of  the  set  (although  it  grows  exponentially  with  dimension).  In 
addition,  a  set  of  sophisticated  numerical  techniques  to  accurately  solve  PDEs 
may  be  drawn  upon  for  computation.  In  the  remainder  of  this  section,  we  focus  on 
the  representation  (2),  and  assume  that  the  modeler  can  compute  the  appropriate 
optimization  over  inputs  in  (1)  if  given  x  and  VJ{x,t). 

3.2  Solving  the  Hamilton- Jacobi  PDE 

The  HJ  PDE  (2)  is  well  known  to  have  complex  behavior.  Even  with  smooth 
initial  data  J{x,0)  and  continuous  Hamiltonian  i/(a:,  VJ),  the  solution  J{x,t) 
can  develop  discontinuous  derivatives  in  finite  time;  consequently,  classical  in¬ 
finite  time  solutions  to  the  PDE  are  generally  not  possible.  In  the  quest  for  a 
unique  weak  solution  Crandall  and  Lions  introduced  the  concept  of  the  viscosity 
solution  [17],  which  has  since  been  shown  to  be  the  appropriate  weak  solution 
for  Hamilton- Jacobi-Bellman  type  control  problems  such  as  (1)  (see,  for  exam¬ 
ple,  [18]).  For  most  problems  of  interest,  finding  the  analytic  viscosity  solution 
is  not  possible,  and  so  we  seek  a  numerical  solution. 

Floating  point  arithmetic  and  the  truncation  required  by  finite  series  expan¬ 
sions  conspire  to  ensure  that  any  numerical  approximation  of  the  solution  of  a 
differential  equation  will  contain  errors.  The  algorithms  presented  in  [7]-[16]  seek 
guaranteed  overapproximations  (and  in  some  cases,  underapproximations)  of  the 


Validating  a  Hamilton- Jacobi  Approximation  to  Reachable  Sets  423 

system’s  reachable  sets.  Numerical  methods  for  solving  PDEs,  on  the  other  hand, 
have  traditionally  aimed  for  convergent  approximations:  those  approximations 
that  will  become  exact  as  some  parameter  of  the  method — the  grid  spacing  Ax, 
for  example — goes  to  zero.  While  guaranteed  over  approximation  has  its  pros  and 
cons  for  use  in  reachability  applications,  we  have  decided  to  focus  first  on  con¬ 
vergent  approximations  of  (2)  in  order  to  take  advantage  of  existing  schemes  and 
numerical  analyses  [19,20,21,22,23].  We  can  develop  confidence  in  a  convergent 
approximation’s  accuracy  by  successive  refinement  of  Ax. 

If  we  are  willing  to  pursue  convergent  numerical  approximations  of  (2),  a 
reasonable  question  is  whether  it  would  be  simpler  and  as  reliable  to  solve  for 
the  optimal  trajectories  starting  from  points  on  the  boundary  of  the  initial  set, 
and  thereby  approximate  the  boundary  of  the  reachable  set.  This  technique, 
however,  is  equivalent  to  solving  the  PDE  by  the  characteristic  method,  and 
the  characteristics  of  the  Hamilton- Jacobi  equation  are  known  to  collide  and/or 
separate  [18],  which  would  make  for  an  incorrectly  represented  reachable  set.^ 

Returning  to  methods  of  solving  (2)  numerically,  the  state  space  over  which 
we  compute  reachability  is  topologically  simple,  and  so  we  approximate  the 
solution  of  (2)  on  a  Cartesian  grid  of  nodes.  Three  terms  in  the  equation  must 
be  approximated  at  each  node,  based  on  the  values  of  the  level  set  function  at 
that  node  and  its  neighbors:  the  gradient  VJ,  the  Hamiltonian  if,  and  the  time 
derivative  .  We  discuss  each  of  these  separately. 

In  each  dimension  at  each  grid  point  there  exist  both  left  and  right  approxi¬ 
mations  of  the  gradient  VJ,  depending  on  which  neighboring  grid  points’  values 
are  used  in  the  finite  difference  calculation.  We  label  the  vector  of  left  approxi¬ 
mations  VJ~,  the  vector  of  right  approximations  VJ"^,  and  will  see  below  that 
VJ",  VJ"^  or  some  combination  of  the  two  will  be  used  to  compute  the  numer¬ 
ical  Hamiltonian  ff.  The  accuracy  of  a  derivative  approximation  is  measured 
in  terms  of  the  order  of  its  local  truncation  error;  an  order  p  method  has  er¬ 
ror  II VJ  -  VJ^^II  =  0{Ax'P).  At  the  current  time,  we  have  implemented  the 
basic  first  order  accurate  approximation  for  speed  [21]  and  a  weighted,  essen¬ 
tially  non-oscillatory  fifth  order  accurate  approximation  for  high  fidelity  [20,22]. 
“Non-oscillatory”  in  this  context  indicates  that  near  discontinuities  in  the  level 
set  derivative,  a  scheme  may  revert  to  lower  order  accuracy  so  as  to  avoid  intro¬ 
ducing  spurious  numerical  oscillations  into  the  solution.  Technically,  therefore, 
all  schemes  are  globally  first  order  accurate,  but  in  practice  the  higher  order 
accuracy  in  the  smooth  parts  of  the  solution  produces  better  global  results.  This 
property  is  sometimes  called  “high  resolution”  to  distinguish  it  from  true  high 
order  accuracy. 

We  have  chosen  to  use  the  well  studied  Lax-Friedrichs  numerical  Hamiltonian 
approximation  H  [20,24] 

H{x,  V  J-,  V  J+)  =  H(x,  YJZ±EJl)  _  ia'^(V  J+  -  VJ”),  (3) 

^  For  example,  it  turns  out  that  much  of  the  helical  bulge  of  the  reach  set  computed 
in  Section  3.4  lies  on  a  collection  of  optimal  trajectories  fanning  out  from  a  single 
point  on  the  boundary  of  the  problem’s  initial  conditions. 
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where  H{x,  VJ)  is  given  by  (2)  and  the  term  containing  the  vector  coefficient  a 
is  a  high  order  numerical  dissipation  added  to  damp  out  spurious  oscillations  in 
the  solution.  Upwinded  numerical  Hamiltonians  were  considered;  but  although 
they  do  not  require  the  artificial  dissipation  of  Lax-Friedrichs,  they  cannot  easily 
deal  with  the  VJ  dependent  flow  appearing  in  (2). 

The  time  derivative  of  the  PDE  is  handled  by  the  method  of  lines:  the  value 
of  the  level  set  function  J  at  each  node  is  treated  as  an  ODE  ^  =  H,  with 
H  given  by  (3).  General  ODE  solvers,  such  as  Runge-Kutta  (RK)  schemes,  can 
then  be  applied.  The  explicit  nature  of  these  techniques,  however,  limits  the  size 
of  the  timestep  to  some  flow  speed  dependent  multiple  of  the  grid  spacing — 
typically  a  small  fraction — called  the  Courant-Priedrichs-Lewy  (CFL)  number. 
Standard  RK  iterations  lead  to  very  small  CFL  values  and  can  introduce  spurious 
oscillations  into  a  numerical  Hamilton- Jacobi  solution;  therefore,  we  use  total 
variation  diminishing  (TVD)  versions  of  Runge-Kutta  (see,  for  example,  [19,23]). 
We  have  currently  implemented  TVD  RK  schemes  which  are  first  and  second 
order  accurate  in  time.  Due  to  CFL  restrictions  the  timestep  is  usually  much 
smaller  than  the  grid  spacing,  so  it  is  possible  to  use  lower  order  accuracy  in 
time  than  in  space  without  noticeable  loss  of  solution  quality. 

3.3  Localizing  Computation 

The  Hamilton- Jacobi  equation  (2)  describes  the  evolution  of  the  level  set  func¬ 
tion  over  all  of  space.  But  we  are  only  interested  in  its  zero  level  set;  thus,  we 
can  restrict  our  computational  updates  to  nodes  near  the  boundary  between 
positive  and  negative  J{x,t) — an  idea  variously  called  “local  level  sets”  [25]  or 
“narrowbanding”  [21].  We  have  implemented  a  new  variant  of  this  method  in 
our  code. 

Because  the  boundary  is  of  one  dimension  less  than  the  state  space,  consider¬ 
able  savings  are  available  for  two  and  three  dimensional  problems.  If  the  number 
of  nodes  in  each  dimension  is  n  (proportional  to  Ax~'^)  and  the  dimension  d,  the 
total  number  of  nodes  is  the  CFL  restriction  on  timestep  means  that  total 

computational  cost  is  With  local  level  sets,  we  reduce  computational 

costs  back  down  to 

3.4  Numerical  Validation  of  Aircraft  Collision  Avoidance 

The  numerical  schemes  mentioned  above  for  solving  the  Hamilton- Jacobi  equa¬ 
tion  are  complicated;  therefore,  it  is  not  surprising  that  theoretical  proofs  of 
convergence  to  the  viscosity  solution  are  available  for  only  the  very  simplest  low 
order  accuracy  methods  [24].  High  resolution  methods  have  instead  been  sub¬ 
jected  to  “numerical  validation”:  comparison  to  known  analytic  solutions  and 
lower  order  accurate  approximations  of  an  extensive  collection  of  examples  for  a 
broad  range  of  grid  sizes  [20],  from  which  can  be  drawn  encouraging  conclusions 
regarding  their  accuracy. 

In  this  section  we  present  a  similar  validation  of  our  implementation  on  the 
single  mode,  three  dimensional  aircraft  collision  avoidance  example  (see  [3,2]  for 
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(5,2)  scheme  on  100^  grid 


(1,1)  scheme  on  50^  (inner),  100^  (middle) 
and  200^  (outer)  grids 


10  12  14  16  18 


Fig.  2.  Reachable  Set  for  Aircraft  Collision  Avoidance  Example 


details).  The  example  features  a  control  aircraft  trying  to  avoid  collision  with 
a  disturbance  aircraft,  where  both  aircraft  have  fixed  and  equal  altitude,  speed 
and  turning  radius — they  may  only  choose  which  direction  they  will  turn: 

Xr  =  -Vu  +  Vd  cos  'Ipr  "h  Uyr,  Vr  =  Vd  Sm  'Ipr  “  UXr,  Ipr  d  -  U, 

where  Vu  =  Vd  —  ^  are  the  aircraft  speeds,  Xr  and  yr  are  the  relative  planar 
location  of  the  aircraft  and  'tpr  is  their  relative  heading.  The  inputs  |u|  <  1 
and  |d|  <  1  are  the  control’s  and  disturbance’s  respective  turn  rates.  The  initial 
unsafe  set  J(x,  0)  is  the  interior  of  the  radius  five  cylinder  centered  on  the  'ipr  axis. 
Choosing  optimal  inputs  according  to  (1)  with  uq  =  i/max  =  d  and  =  z/min  = 
we  get  the  optimal  Hamiltonian: 

H{x,p)  —  -piVu  Tpi'^dCOS'^r  +P2^ciSin^r-  +  \p\yr  -  P^^r  -  Pz\  ”  IPsI* 

Using  our  new  C++  implementation,  grid  sizes  corresponding  to  50,  70,  100, 
140,  and  200  nodes  in  each  dimension  were  tried  with  a  low  order  accurate 
scheme  (first  order  space  and  time,  hereafter  referred  to  as  the  “(1)1)”  scheme) 
and  a  high  resolution  scheme  (fifth  order  space  and  second  order  time,  hereafter 
the  “(5,2)”  scheme).  On  the  eight  million  node  finest  grid — only  around  10% 
of  which  is  being  actively  updated  on  any  one  timestep  by  the  local  level  set 
algorithm — execution  time  for  the  (5,2)  scheme  was  about  eighteen  hours  on  a 
Sun  UltraSparc  II  with  lots  of  memory.  Reducing  the  grid  size  in  half  results 
in  the  expected  eightfold  savings  in  memory  and  time;  hence,  the  coarsest  grid 
takes  only  fifteen  minutes  with  the  (5,2)  scheme. 
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Fig.  3.  Convergence  of  (5,2)  Scheme  to  Finest  Grid  Solution  (Jn  is  the  solution  J{x,t) 
on  a  grid  size  of  n) 


Results  are  visualized^  by  the  zero  level  isosurface  of  the  unsafe  reachable  set 
G,  shown  in  Figure  2.  On  the  left  is  a  head-on  view  of  the  (5,2)  solution.  On  the 
right  is  a  zoomed  overhead  view  of  the  point  of  the  bulge  computed  by  the  (1,1) 
scheme  for  several  grid  sizes.  The  fact  that  the  solutions  grow  closer  together  as 
the  grid  is  refined  provides  visual  evidence  of  convergence. 

The  solutions  produced  by  the  (5,2)  scheme  are  visually  identical  for  all 
grids,  and  to  show  quantitative  convergence  as  the  grid  is  refined  we  require 
a  suitable  error  metric.  Comparing  the  value  of  J{x,t)  over  the  entire  domain 
is  inappropriate,  since  our  algorithms  assume  that  we  seek  only  an  accurate 
computation  of  its  zero  level  set.  Instead,  we  consider  just  the  nodes  neighboring 
the  zero  level  set — those  nodes  which  have  at  least  one  adjacent  node  whose  J 
value  is  of  opposite  sign.  We  compare  solutions  on  the  four  coarser  grids  to 
the  solution  on  the  finest  grid,  using  linear  interpolation  on  the  finest  grid  if 
necessary.  Figure  3  demonstrates  that  the  scheme  is  converging  to  the  finest  grid’s 
solution  of  (2)  at  approximately  a  linear  rate  in  both  average  error  and  pointwise 
maximum  error.  We  cannot  expect  to  show  a  higher  order  convergence  rate 
because  of  the  linear  interpolation  used  to  evaluate  the  error  and,  as  explained 
in  Section  3.2,  the  scheme  is  truly  high  order  accurate  only  in  smooth  portions 
of  the  solution. 

Two  conclusions  can  be  drawn  from  Figures  2  and  3.  First,  low  order  schemes 
are  not  at  all  competitive  in  terms  of  accuracy  with  the  (5,2)  scheme.  Thus,  while 
our  previously  reported  best  results  [2]  took  only  an  hour  to  run  in  Matlab, 
because  they  used  a  (slightly  different)  first  order  scheme,  our  new  (5,2)  imple¬ 
mentation  can  produce  more  accurate  results  in  about  fifteen  minutes  using  only 
the  coarsest  grid.  Second,  the  pointwise  maximum  error  of  the  (5,2)  scheme  is 
always  less  than  the  grid  spacing,  so  if  a  50~^  =  2%  error  is  tolerable  for  this 
application,  only  this  fastest,  coarsest  grid  need  ever  be  run. 

^  Figure  2  and  Figure  6  visualize  some  level  set  surfaces  as  triangular  meshes;  these 
are  not  the  meshes  on  which  the  Hamilton- Jacobi  PDF  was  solved,  but  rather  an 
artifact  of  three  dimensional  Matlab  visualization  techniques. 
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4  Aircraft  Landing  Example 


Once  a  method  of  determining  continuous  reachability  is  available,  the  discrete 
iteration  of  the  algorithm  described  in  Section  2  is  relatively  straightforward. 
In  fact,  for  discrete  transition  graphs  with  no  cycles  it  is  possible  to  order  the 
continuous  reachability  problems  such  that  no  discrete  iteration  is  required  (e.g. 
the  three  mode  example  presented  in  [2]).  In  order  to  examine  the  complications 
induced  by  discrete  cycles — such  as  how  to  avoid  zenoness,  in  what  order  to  exe¬ 
cute  the  continuous  reachability  problems,  and  how  to  determine  which  switches 
are  active — a  new  example  has  been  developed,  which  exhibits  those  difficulties 
and  has  real  life  applications:  the  landing  of  a  civilian  airliner. 

Physical  model:  A  simple  point  mass  model  for  aircraft  vertical  navigation 
is  used,  which  accounts  for  lift  L,  drag  D,  thrust  T,  and  gravity  mg  (see  [3]  and 
references  therein).  State  variables  are  aircraft  height  2;,  horizontal  position  x, 
velocity  V  =  and  flight  path  angle  7  =  tan“^(f).  Inputs  are  thrust 

T  and  angle  of  attack  a,  where  aircraft  pitch  0  =  7  -f-  a  (see  the  left  side  of 
Figure  4).  The  equations  of  motion  can  be  expressed  as  follows: 


A 

dt 


V 

7 

X 

z 

~  [T  cos  a  -  D{a,  V)  -  mg  sin  7] 
^  [T  sin  a  -h  L{a,  V)  —  mg  cos  7] 

V  cos  7 

V  sin  7 


(4) 


The  functions  L(a,  V)  and  D{a,V)  are  modelled  based  on  empirical  data  [26] 
and  Prandtl’s  lifting  line  theory  [27]: 


L{a,  V)  =  ^pSV^CL{a),  D{a,  V)  =  \pSV^CD{oi), 

where  p  is  the  density  of  air,  S  is  wing  area,  and  C[^{a)  and  Cd(q;)  are  the 
dimensionless  lift  and  drag  coefficients. 

In  determining  Cl  (q;)  we  will  follow  standard  auto-lander  design  and  assume 
that  the  aircraft  switches  between  three  fixed  flap  deflections  =  0°,  ^  =  25° 
and  6  =  50°  (with  slats  either  extended  or  retracted),  thus  constituting  a  hybrid 
system  with  different  nonlinear  dynamics  in  each  mode.  This  model  is  represen¬ 
tative  of  current  aircraft  technology;  for  example,  in  Airbus  cockpits  the  pilot 
uses  a  lever  to  select  among  four  predefined  flap  deflection  settings.  We  assume  a 
linear  form  for  the  lift  coefficient  C'x,(q:)  =  /i5-H4.2q:,  where  parameters  h^o  =  0.2, 
/1250  —  0.8  and  =1.2  are  determined  from  experimental  data  for  a  DC9- 
30  [26].  The  value  of  a  at  which  the  vehicle  stalls  decreases  with  increasing  flap 
deflection:  =  16°,  =  13°,  =  11°;  slat  deflection  adds  7°  to  the 

Q,max  •  mode.  The  right  side  of  Figure  4  gives  a  graphical  summary  of  the 

possible  configurations.  The  drag  coefficient  is  computed  from  the  lift  coefficient 
as  [27]  Cnia)  —  0.041  +  0.0450^  (a)  and  includes  flap  deflection,  slat  extension 
and  gear  deployment  corrections.  So  for  a  DC9-30  landing  at  sea  level  and  for 
all  a  e  [-5°,q:J'^^],  the  lift  and  drag  terms  in  (4)  are  given  by 

L{a,  V)  =  68.6  {hg  +  4.2a)V'^  D(a,  V)  =  (2.81  +  3.09  (hg  +  4.2a)*)F^  (5) 
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Fig.  4.  Left:  Force  diagram  for  the  point  mass  approximation  of  the  aircraft.  Right: 
lift  coefficient  CL(a)  model  for  the  DC9-30  [26].  Circles  located  at 
indicate  the  stall  angle  and  the  corresponding  lift  coefficient  in  each  mode. 


Flap  deflection  dynamics  model:  In  reality,  the  decision  to  move  from 
one  deflection  setting  to  another  can  occur  at  any  time,  but  approximately  10 
seconds  are  required  for  a  25°  degree  change  in  flap  deflection.  A  five  state  model 
of  this  situation  is  shown  on  the  left  side  of  Figure  5,  where  the  system  is  in  state 
R  if  the  flaps  are  retracting  and  state  D  if  the  flaps  are  deflecting.  The  system 
is  zeno  because  instantaneous  switches  are  allowed  between  any  modes. 

Current  implementation:  For  our  preliminary  implementation,  we  have 
chosen  to  ignore  the  continuous  dynamics  associated  with  discrete  mode  switch¬ 
ing,  allowing  the  flaps  and  slats  to  move  instantly  to  their  commanded  positions. 
However,  if  such  instantaneous  controlled  switches  were  always  enabled  then  the 
system  would  be  zeno;  therefore,  we  introduce  transition  modes  Ot,  2bt  and  50t, 
which  use  the  envelopes  and  flight  dynamics  of  the  regular  modes  Ow,  25d  and 
50d  (the  discrete  automaton  is  shown  on  the  right  side  of  Figure  5) .  A  regular 
mode  may  make  a  controlled  switch  to  a  transition  mode,  so  flight  dynamics  can 
be  changed  instantly.  Transition  modes  have  only  a  timed  switch  at  t  =  tdeiay,  so 
controlled  switches  will  be  separated  by  at  least  tdeiay  time  units  and  the  system 
is  nonzeno.  For  the  executions  shown  below,  tdeiay  =  0.5  seconds. 

Landing:  Extensive  descriptions  of  the  final  stage  of  landing,  when  aircraft 
height  is  below  50  feet,  exist  (see,  for  example,  [26,28]).  Restrictions  on  the  flight 
path  angle,  aircraft  velocity  and  touchdown  (TD)  speed  are  used  to  determine 
the  initial  safe  set  Eq: 


(z<0 

V  > 

V  < 

sin  7  >  io 

[7<0 


landing  or  has  landed 
faster  than  stall  speed 
slower  than  limit  speed 
limited  TD  speed 
monotonic  descent 


2:  >  0  aircraft  in  the  air 

faster  than  stall  speed 
slower  than  limit  speed 
7  ^  limited  descent  flight  path 

7^0  monotonic  descent 


(6) 


We  again  draw  on  numerical  values  for  a  DC9-30  [26]:  stall  speeds  =  78 

~  =  58  m/s,  maximal  touchdown  speed  zq  =  0.9144 


Validating  a  Hamilton- Jacobi  Approximation  to  Reachable  Sets 


429 


Fig.  5.  Discrete  transition  graph  of  slat  and  flap  settings.  The  left  graph  shows  the 
model  with  flap  deflection  dynamics  and  the  right  graph  shows  the  currently  imple¬ 
mented  model.  Solid  lines  are  controlled  switches  ((Te  in  this  version  of  the  reachability 
problem)  and  dashed  lines  are  uncontrolled  switches  (tJc)- 


m/s,  and  maximal  velocity  83  m/s.  For  passenger  comfort,  the  aircraft’s 

input  range  is  restricted  to  T  G  [0  kN,  160  kN]  and  a  G  [0°,  10°]. 

The  interior  of  the  surface  shown  in  the  first  row  of  Figure  6  represents  Eq 
for  each  mode.  The  second  row  of  the  figure  shows  the  safe  envelope  E  when 
there  is  no  mode  switching.  Portions  of  Eq  are  excluded  from  E  for  two  reasons. 
States  near  z  =  0  correspond  to  low  altitudes  and  are  too  close  to  the  ground  at 
steep  flight  path  angles  to  allow  control  inputs  time  to  prevent  the  plane  from 
crashing.  States  close  to  the  stall  velocity  correspond  to  low  speeds  where  there 
is  insufficient  lift  and  the  flight  path  angle  becomes  steeper  than  that  allowed 
by  the  flight  envelope.  This  latter  condition  holds  throughout  the  very  narrow 
range  of  speeds  allowed  in  mode  Ow,  with  the  result  that  only  post- touchdown 
states  (z  <  0)  are  controllable  in  this  mode.  The  third  row  shows  how  E  can  be 
increased  if  switches  are  permitted  (for  example,  mode  Ow  becomes  completely 
controllable).  Mode  50d  is  the  best  to  be  in  for  landing  and  there  is  no  difference 
in  E  with  or  without  switching  enabled.  The  fourth  row  shows  slices  of  the  set 
in  the  third  row,  taken  at  z  =  3  meters.  The  light  grey  regions  are  unsafe  G  and 
the  dark  grey  are  safe  E.  The  figure  shows  that  modes  Ou  and  25d  are  safe  only 
because  there  exists  a  discrete  switch  to  a  safe  state  in  another  mode. 

We  have  presented  and  numerically  validated  a  tool  for  determining  accurate 
approximations  of  reachable  sets  for  hybrid  systems  with  nonlinear  continuous 
dynamics  and  adversarial  continuous  and  discrete  inputs.  By  developing  conver¬ 
gent  approximations  of  such  complex  systems,  we  will  be  better  able  to  synthesize 
aggressive  but  safe  controllers.  As  an  example,  the  six  mode  auto-lander  shows 
that  for  envelope  protection  purposes  the  safest  control  decisions  are  to  switch 
directly  to  full  flap  deflection,  but  to  maintain  airspeed  until  touchdown.  With 
the  summary  data  from  the  reachability  analysis,  such  decisions  can  be  made 
based  on  local  state  information;  without  it  the  auto-lander  may  not  detect  that 
low  speeds — while  still  within  the  flight  envelope — lead  inevitably  to  unsafe  flight 
path  angles. 


Fig.  6.  Maximally  controllable  safe  envelopes  for  the  multimode  landing  example.  Prom 
left  to  right  the  columns  represent  modes  On,  25d  and  50<i. 
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Our  current  work  includes  further  validation  of  our  numeric  algorithm,  ex¬ 
tending  our  implementation  to  four  continuous  dimensions  in  order  to  capture 
the  full  landing  example  dynamics,  projections  to  capture  higher  dimensional  dy¬ 
namics,  schemes  for  over  approximating  the  solution  of  the  HJ  PDE,  automation 
of  the  discrete  algorithm,  and  parallel  implementations. 
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Abstract.  In  this  paper,  we  formulate  and  robustly  solve  a  quite  gen¬ 
eral  class  of  hybrid  controller  synthesis  problems.  The  type  of  controller 
we  investigate  is  the  switching  control  mechanism  of  a  hybrid  automaton 
(via  guard  and  mode  invariant  sets),  and  the  robustness  result  is  with 
respect  to  variations  in  the  right  hand  sides  of  the  differential  equations 
that  depend  continuously  on  a  parameter.  We  present  a  novel  method¬ 
ology  for  controller  design  and  synthesis  which  uses  modal  logic  as  a 
formalism  for  reasoning  about  sets  of  plant  states,  and  various  operators 
on  sets  arising  from  the  differential  equations  and  from  metric  tolerance 
relations  on  the  state  space. 


1  Introduction 

In  general  terms,  a  hybrid  system  H  can  be  said  to  satisfy  a  performance  spec¬ 
ification  robustly  if  every  system  H'  in  some  nominated  variation  class  around 
H  also  satisfies  that  specification.  Likewise,  a  synthesis  procedure  for  a  class 
of  control  problems  can  be  called  robust  if  the  nominal  closed-loop  hybrid  sys¬ 
tem  obtained  from  the  solution  controller  can  be  shown  to  robustly  satisfy  each 
of  the  specifications  of  the  problem,  with  respect  to  some  nominated  variation 
class.  Robustness  in  hybrid  control  systems  is  an  under-explored  topic.  A  start¬ 
ing  point  is  given  in  [10],  which  proposes  a  range  of  variation  classes  for  hybrid 
automata,  including  near  relatives  of  those  in  the  present  work  and  its  predeces¬ 
sor  [6].  Robustness  issues  for  hybrid  controller  design,  for  a  variety  of  different 
control  settings  and  problems,  are  also  investigated  in  [3,8,19,21]. 

In  this  paper  we  find  a  robust  solution  to  a  rather  general  switching  control 
problem  for  hybrid  systems.  The  plant  consists  of  a  finite  number  of  continuous 
systems,  given  by  differential  equations  over  a  common  state  space;  the  con¬ 
troller  steers  the  plant  state  by  determining  when  to  discretely  switch  between 
the  various  differential  equations;  and  the  closed-loop  trajectories  correspond 
to  those  of  (a  subclass  of)  the  widely  accepted  hybrid  automaton  model.  In 
addition  to  the  well-studied  classes  of  safety  (reachability  or  invariance)  and 

*  Research  partially  supported  by  US  Office  of  Naval  Research,  Grant  N  00014-98-1- 
0535. 


M.D.  Di  Benedetto,  A.  Sangiovanni-Vincentelli  (Eds.):  HSCC  2001,  LNCS  2034,  pp.  433-446,  2001. 
(c)  Springer- Verlag  Berlin  Heidelberg  2001 


434  T.  Moor  and  J.  M.  Davoren 


liveness  (non-blocking  and  non- Zeno)  performance  specifications,  we  deal  with 
a  class  of  event  sequence  specifications,  requiring  that  trajectories  traverse  in 
prescribed  sequences  through  the  blocks  of  a  given  finite  partition  of  the  plant 
state  space.  This  gives  a  general-purpose  way  of  specifying  the  attainment  of 
local  goals  along  hybrid  trajectories,  and  integrating  the  type  of  event  sequence 
specifications  examined  in  DBS  approaches  to  hybrid  systems  [5,12,17]. 

In  [6] ,  we  develop  an  abstract  algorithm  which  solves  this  controller  synthesis 
problem  for  arbitrary  differential  equations  with  unique  solutions,  with  a  proof 
of  finite  termination  derived  from  an  assumption  of  compactness  of  the  sets  given 
in  the  data  of  the  specifications.  In  that  work,  we  consider  one  type  of  variation 
class  that  is  motivated  by  considerations  of  sensor  and  actuator  imprecision,  and 
is  obtained  by  allowing  a  metric  tolerance  or  “margin  of  error”  around  the  guard 
sets  and  in  the  reset  relations;  we  have  shown  that  our  synthesis  procedure  is 
robust  with  respect  to  that  class.  In  the  present  paper,  we  turn  our  attention 
to  the  more  traditional  control-theoretic  perspective  on  robustness  in  terms  of 
parameter  uncertainty’,  i.e.  variations  in  the  right  hand  sides  of  the  differential 
equations  that  depend  continuously  on  a  parameter.  While  these  two  variation 
classes  are  quite  distinct,  a  key  technical  tool  for  both  cases  are  metric  tolerance 
relations,  which  are  put  to  use  in  different  ways. 

This  paper  also  demonstrates  the  flexibility  and  adaptability  of  our  novel 
methodology  for  hybrid  controller  synthesis  based  on  modal  logic,  first  developed 
in  [6,7].  For  our  purposes,  modal  logic  is  best  viewed  as  a  formalism  for  reasoning 
about  sets  of  states  and  operators  on  sets  arising  from  relations  on  the  state 
space.  Considered  as  a  family  of  logics,  modal  logic  includes  the  temporal  logics 
more  commonly  used  in  formal  verification  of  hybrid  systems.  More  precisely, 
we  work  with  a  polymodal  fusion  of  several  normal  monomodal  logics  [20].  The 
main  benefits  our  methodology  are  the  following. 

•  Modal  logic  provides  us  with  a  uniform  framework  for  investigating  not  only 
the  widely  used  pre-  and  post-image  operators  induced  by  continuous  flows, 
but  also  operators  induced  by  metric  tolerance  relations,  and  the  latter  are 
essential  in  the  context  of  robustness.  As  distinct  from  temporal  logics,  we 
reason  about  the  component  parts  of  hybrid  trajectories,  and  this  is  essential 
for  synthesis  as  opposed  to  analysis  of  hybrid  systems. 

•  We  use  modal  logic  not  merely  as  a  convenient  notation,  but  also  draw  on 
the  power  of  deductive  proof  systems.  In  the  course  of  proving  the  correctness 
of  our  synthesis  algorithm,  we  show  that  certain  key  modal  formulas  are  for¬ 
mally  deducible  from  the  statement  of  the  algorithm  together  with  explicit 
assumptions;  this  appeals  to  the  soundness  of  a  suitable  Hilbert  proof  system 
w.r.t.  the  Kripke  (transition  system)  semantics.  In  future  work  we  will  employ 
automated  reasoning  tools  based  on  the  decidability  of  the  logical  consequence 
and  validity  problems  for  modal  logics,  utilising  tableaux  proof  systems  [9]. 

•  In  our  use  of  modal  logic,  we  make  a  clean  separation  between  (i)  determining 
what  sets  need  to  be  computed  in  order  to  solve  the  synthesis  problem,  and 
(ii)  how  and  when  such  computations  can  be  performed  effectively.  Issue  (i) 
is  resolved  by  the  our  synthesis  algorithm  below.  Issue  (ii)  is  essentially  the 
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standard  model  checking  problem  for  hybrid  systems,  and  any  model  checking 
tools  —  either  exact  [1,2,14,15]  or  approximate  [4,13,16]  —  can  be  used  to 
implement  our  synthesis  algorithm. 

In  this  short  paper,  we  restrict  our  focus  to  the  core  ingredients,  and  to  those 
aspects  of  the  work  that  are  crucial  for  plant  parameter  robustness.  Consult  [6] 
for  a  more  detailed  account  of  our  framework  based  on  modal  logic. 

The  body  of  the  paper  is  organised  as  follows.  In  Section  2,  we  briefly  review 
hybrid  automata,  define  plant  parameter  variation  classes,  and  give  a  key  result 
on  parameterised  vector  fields.  In  Section  3,  we  formally  state  the  controller 
synthesis  problem.  Section  4  is  a  terse  review  of  modal  logic  applied  to  hybrid 
systems,  and  in  Section  5,  we  give  our  abstract  synthesis  algorithm,  formalised 
in  the  language  of  modal  logic.  In  Section  6,  we  outline  the  proof  of  the  main 
result  of  robust  correctness.  The  concluding  Section  7  includes  a  brief  discussion 
of  effective  implementations  of  the  procedure. 


2  Hybrid  Automata 

We  work  with  the  standard  and  widely  accepted  hybrid  automaton  model  of 
Alur,  Henzinger  et  al  [1,2]. 

Definition  1.  A  hybrid  automaton  is  a  system 

H  =  [  Q,  Ej  X,  {Eq,  1  ^^^q,q'^ {q,q’)^E  )  5  (1) 

where:  Q  is  a  finite  set  of  discrete  control  modes;  E  C  Q  x  Q  is  the  discrete 
transition  relation;  A  C  the  continuous  state  space;  for  each  q  £  Q, 

Fq  :  X  ^  is  a  vector  field,  and  InVq  C  X;  and  for  each  (q,q')  €  E, 
f’q,q'  Q  X  X  X  is  a  reset  relation,  and  Grdq^q>  —  dom(rg^g/). 

In  order  to  ensure  that  closed-loop  trajectories  are  well-defined,  we  assume 
that  the  vector  fields  Fq  are  locally  Lipschitz  continuous,  and  the  state  space 
X  is  open.  Then  from  each  initial  condition  xq  e  X,  each  differential  equation 
X  =  Fq{x)  has  a  unique  maximal  integral  curve  in  X  on  a  well  defined  maximal 
interval  of  time  [0,  Tq{xo)),  where  Tq{xo)  €  U  {oo}.  We  denote  this  maximal 
curve  by 


d>q{xo,  •):  [0,Tq{xo))^X.  (2) 

In  the  case  of  Tq{xo)  <  oo,  it  is  well  known  that  ^q{xo,  •)  escapes  from  any 
bounded  subset  of  X  at  some  time  less  than  or  equal  to  Tq{xo).  For  the  scope 
of  this  paper,  we  can  restrict  attention  to  bounded  invariant  sets  InVq.  Then 
maximal  curves  from  xq  G  InVq  either  leave  InVq  within  finite  time  or  stay 
within  InVq  forever  with  Tq{xQ)  =  oo.  Closed-loop  trajectories  are  then  defined 
as  follows. 

Definition  2.  A  trajectory  of  a  hybrid  automaton  H  is  a  finite  or  infinite  se¬ 
quence  T)  =  {Ai,qi,ji)i^j  such  that  for  each  i  €  I: 
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•  the  duration  Ai  G  M”*”  U  {oo},  with  Ai  =  oo  only  if  I  is  finite  and  i  =  max(/); 

•  the  discrete  state  Qi  £  Q; 

•  the  continuous  curve  ji  :  [0,  A]  X  satisfies -fi{t)  =  ^^.(7^(0),^)  and^i{t)  G 
Invq^  for  all  t  G  [0,  Ai],  with  the  convention  that  [0,  A^]  is  [0,  00)  if  Ai  =  00; 

•  ifi  <  sup(/),  then  {q^qi+i)  G  E  and  ^i{Ai)  7i+i(0)- 

A  trajectory  will  be  called:  step-infinite  if  it  makes  infinitely  many  switches; 
time-infinite  if  the  sum  over  all  durations  is  unbounded;  and  full  if  it  is  either 
step-infinite  or  time-infinite  or  else  it  is  blocked,  in  the  sense  that  it  cannot  be 
extended  to  reach  any  further  guard  region. 

A  broad  framework  of  variation  classes  for  hybrid  automata  is  proposed  in 
[10].  Our  interest  here  is  in  parameter  variations  in  the  vector  fields. 

Definition  3.  Given  a  hybrid  automaton  H  as  in  Eq.  (1),  let  :  X  ^ 
be  a  family  of  vector  fields  parameterised  by  the  discrete  modes  q  ^  Q  and  an 
uncertainty  parameter  v  C  where  0  G  V  and  =  Fq.  Then 

~  {Q^  F^  X,  {Fq  ,InVq}q^Q,  {rq,q'yGrdq^q>}(^qqf'^^E  )  ,  (3) 

|H|<e}  (4) 

defines  a  parameterised  variation  class  around  the  nominal  model  =  H  with 
variation  bound  e. 

In  correspondence  with  the  nominal  model,  we  denote  the  maximal  integral 
curves  of  the  vector  field  by  •):  (^o))  ^  X  where  T^{xq)  G 

R+  U  {00} .  The  following  assumptions  on  the  vector  fields  are  to  ensure  that  the 
flow  ^q{xQ,  t)  is  continuous  in  v  and  xq. 

(AO)  The  parameter  set  V  is  open.  The  vector  field  Fq{x)  is  continuous  in  both 
X  and  V.  Furthermore,  F^(x)  is  locally  Lipschitz  continuous  in  x  uniformly 
in  i.e.  there  exists  a  Lipschitz  constant  which  may  depend  on  x  but  not 
on  V. 

In  particular,  assumption  (AO)  ensures  that  for  any  given  finite  time  interval 
and  any  given  open  tube  around  the  nominal  integral  curve  ^q{xQ,t),  all  varia¬ 
tions  ^J(a:o,  t)  evolve  within  that  tube  -  provided  that  the  variation  is  sufficiently 
small;  e.g.  [11],  Theorem  2.6.  In  the  hybrid  setting,  we  need  to  examine  contin¬ 
uous  parameter  dependency  w.r.t.  a  given  domain  D  in  the  state  space,  rather 
than  w.r.t.  a  given  interval  on  the  time  axis.  That  is,  we  are  interested  in  the 
dependency  of  ^^(2:0,  t)  in  v  as  long  as  that  curve  evolves  within  an  invariant 
set  Invq.  We  formalise  these  ideas  in  terms  of  metric  tolerance  relations,  and  in 
so  doing,  set  up  the  link  to  modal  logics. 

Definition  4.  Given  a  metric  d  on  the  state  space  X,  the  5-ball  Bs{x)  of  radius 
6  >  0  with  centre  x  ^  X  is  defined  by 

Bsix)  {y&X  I  d(a;, y)  <  (5 }  . 


(5) 
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For  a  set  A  C  X,  we  call  the  set  Bs{A)  {x  £  X\  Bs{x)  n  A  ^  0}  the  6- 
expansion  of  A.  We  also  call  the  (reflexive  and  symmetric)  relation  Bs  Q  X  x  X 
a  metric  tolerance  relation.  For  the  scope  of  this  paper,  d  is  assumed  to  be  a 
metric  that  induces  the  standard  Euclidean  topology  on  X. 

For  a  set  ^  C  X,  let 

T^{A,xo)=sup{t<T^(xo)  I  (Vse[0,r))<?-^(xo,s)€^}  (6) 

denote  the  time  at  which  ^g{xo,  ■ )  escapes  from  A,  so  (xq)  =  T^(X,xo). 

Proposition  1.  Let  D  be  a  compact  set  with  B 45(D)  C  X  for  a  given  metric 
tolerance  S  >  0.  Furthermore,  assume  Tq{Bs5(D),Xo)  <  00  for  all  xq  g  D.  Then 
there  exits  a  variation  bound  £  >  0  such  that  Tq{D,xo)  <  Tq{B2s{D),xo)  < 
T^{xo)  and  ^q{xo,  t)  G  B25{^q(^o,  t))  for  all  t  <  Tq(B2s{D),xo),  all  xq  £  D 
and  all  v,  ||u||  <  £. 

Proof  Apply  [11],  Theorem  2.6,  together  with  a  standard  compactness  argu¬ 
ment. 


Fig.  1.  Illustration  of  Proposition  1 


Figure  1  illustrates  a  perturbed  integral  curve  lying  within  a  2(J-tube  around 
the  nominal  curve  from  a  point  xq  £  D,  as  given  by  Proposition  1.  When  a  hybrid 
automaton  with  bounded  invariant  sets  is  designed  so  that  when  an  integral 
curve  leaves  its  invariant  set,  it  does  so  by  some  uniform  minimum  distance, 
Proposition  1  provides  an  elementary  robustness  property  for  this  continuous 
evolution  in  between  any  two  successive  discrete  control  switches.  However,  even 
small  variations  in  the  parameter  may  have  the  effect  that  a  perturbed  trajectory 
runs  into  a  different  guard  set  than  the  corresponding  nominal  trajectory.  In 
turn,  such  a  perturbed  trajectory  may  switch  to  a  different  vector  field  and  thus 
may  potentially  stray  far  away  from  the  nominal  trajectory.  The  avoidance  of 
this  phenomenon  motivates  several  of  the  design  choices  in  formulating  a  robust 
solution  to  our  target  control  problem. 
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3  Control  Problem  Statement 

A  hybrid  automaton  can  be  seen  as  the  closed-loop  feedback  system  resulting 
from  the  inter- connection  of  a  switched  continuous  plant  and  a  discrete  switching 
controller.  See  [6]  for  a  more  detailed  analysis  of  this  control-theoretic  content 
of  a  hybrid  automaton.  For  the  controller  synthesis  problem  under  investigation, 
the  plant  is  given  by  a  finite  family  of  vector  fields  Fc  \  X  ^  indexed  by  a 
control  alphabet  c  6  C.  We  then  ask  for  a  synthesis  procedure  that  constructs  a 
closed-loop  hybrid  automaton  H  by  building  the  missing  entities  that  form  the 
switching  control  mechanism,  namely  Q,  E,  InVg,  and  Grdq^qf,  where  the  reset 
relation  is  required  to  be  elementary]  i.e. 

rq,q'  =  test.Grdq^q>  {  {x,x')  £  X  X  X  \  X  e  Grdq^q>  and  a:'  =  x  }  .  (7) 

As  Q  is  not  known  in  advance,  the  synthesis  procedure  also  needs  to  allocate  a 
particular  control  c  G  C  (indexing  a  vector  field)  to  each  discrete  mode  q  €  Q. 

The  control  goal  is  to  satisfy  the  following  closed-loop  performance  specifi¬ 
cations. 

(51)  Safety:  given  a  proscribed  set  Bad  C  X,  construct  a  set  Good  C  X  ~  Bad 
with  the  property  that  every  iJ-trajectory  starting  in  Good  always  remains 
outside  Bad. 

(52)  Event  sequence  behaviour  with  5-overlaps:  given  a  finite  partition  {Ek}k^K 
of  X  —  J5ad,  a  relation  next  C  K  x  K,  and  a  metric  parameter  (5  >  0, 
let  Ak  =  Bs{Ek)  be  the  ^-expansion  of  the  partition  block  Ek,  for  each 
k  £  K]  the  requirement  is  that  for  every  full  FT-trajectory  starting  in  Good, 
whenever  it  enters  one  of  the  sets  A/t,  it  remains  there  until  it  crosses  into 
Ak'  —  Ak^  for  some  k'  G  next{k). 

(53)  Liveness  I:  every  full  /7-trajectory  starting  in  Good  shall  be  step-infinite. 

(54)  Liveness  II:  every  full  //-trajectory  starting  in  Good  shall  be  time-infinite. 

The  specification  (SI)  is  the  classic  form  of  a  safety  property,  while  (S3)  and 
(S4)  are,  respectively,  the  non-blocking  and  the  non-Zeno  forms  of  liveness  prop¬ 
erties.  The  specification  (S2)  prescribes  an  order  of  traversal  through  the  5- 
expanded  partition  blocks.  Formally,  switches  from  one  such  block  to  another 
are  identified  as  events  from  the  finite  alphabet  K  and  (S2)  requires  the  closed- 
loop  to  generate  a  sublanguage  of  I  Vi  <  sup(/)  :  ki+i  G  next{ki)}.  The 

metric  tolerance  6  ensures  that  the  event  sequence  specification  refers  to  over¬ 
lapping  regions  Ak  H  Ak'  rather  than  the  common  boundaries  bd{Ek)  H  bd{Ek') 
of  partition  blocks.  In  particular,  the  overlaps  are  full  dimensional  and  allow  for 
some  “wiggle  room”  which  is  essential  for  our  robustness  results.  A  more  detailed 
motivation  of  (S2)  is  given  in  [6]. 

Our  synthesis  procedure  is  subject  to  the  following  further  assumptions. 
(Al)  The  set  X  —  Bad  is  compact  (with  respect  to  the  standard  Euclidean 
topology). 

(A2)  The  map  next  C  K  x  K  is  total,  so  for  each  k  ^  K,  there  is  at  least  one 
k'  G  next(k). 

(A3)  For  all  k,k'  G  K  such  that  k  k',  the  partition  blocks  Ek  and  Ek'  are 
contiguous  in  the  sense  that  bd[Ek)  H  bd{Ek')  0. 
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(A4)  For  each  k  e  K,  the  block  Ek  has  a  non-empty  S- contraction]  i.e.  the  set 
{a:  G  X  I  Bs{x)  C  Ek}  is  non-empty. 

( A5)  For  all  k,  k\  k"  G  K  such  that  k  k'  k”,  the  infimum  of  the  metric 
distance  between  points  in  the  set  bd{Ek)  H  bd{Ek>)  and  points  in  the  set 
bd{Ek')  n  bd{Ek")  is  at  least  35. 

By  (Al),  the  relevant  portion  of  the  state  space  is  required  to  be  compact; 
this  is  used  in  applying  Proposition  1  and  in  proving  finite  termination  of  our 
algorithm.  Assumptions  (A2),  (A3)  and  (A4)  are  non-triviality  conditions.  The 
assumption  (A5)  gives  a  foundation  for  non-Zeno-ness  by  ensuring  that  closed- 
loop  trajectories  must  traverse  some  minimum  spatial  distance  when  fulfilling 
the  event  sequence  specification. 

4  Modal  Logics  for  Hybrid  Systems 

This  section  sets  out  only  the  bare  details  of  modal  logics  and  their  application 
to  hybrid  systems.  For  a  more  substantial  account,  the  reader  is  referred  to  [7] 
and  also  to  [6].  The  handbook  chapter  [18]  gives  a  broader  introduction  to  the 
family  of  modal  and  temporal  logics. 

A  modal  signature  is  a  pair  (Rel,  Prp),  where  Rel  is  an  alphabet  of  atomic  re¬ 
lation  labels,  and  Prp  is  an  alphabet  of  atomic  propositions.  The  set  £(Rel,  Prp) 
of  modal  formulas  (p  of  signature  (Rel,  Prp)  is  generated  by  the  grammar: 

(p  ::=  p\^(p\<piy  ‘P2\  {a)v  (8) 

where  p  G  Prp  and  a  G  Rel.  The  other  Boolean  connectives  are  definable,  e.g. 
(^1  A  (p2  “'(^^1  V  ^9^2)5  ^2  — ^  V  (^2):  as  are  the  dual  modal 

operators:  [a](p 

The  formal  semantics  of  modal  (and  temporal)  logics  are  given  with  respect  to 
labeled  transition  systems,  also  called  LTS  models  or  generalized  Kripke  models. 
An  LTS  model  of  signature  (Rel,  Prp)  is  a  structure: 

OT  =  (5,{a®'URe,,{brWp)-  (9) 

where:  S  ^  0  is  the  state  space,  of  arbitrary  cardinality;  for  each  a  G  Rel, 
C  S  X  S  is  a  relation;  and  for  each  p  G  Prp,  C  5  is  a  subset  of 

states.  For  formulas  p  G  £(Rel,  Prp),  the  denotation  set  C  5  is  defined 

by  induction,  starting  with  the  sets  denoting  atomic  propositions  p  G  Prp. 
For  compound  formulas: 


(10) 

[(a)¥.r  Pre^Ca®*)  (bD 

for  a  G  Rel , 

(11) 

y,v<p2r  = 

(12) 

where  the  existential  pre-image  operator  Pre^(r) 
r  C  S'  X  5  is: 

:  V(S)  ^  V{S)  of  a 

relation 

Pre^(r)(A)  {x  G  5  |  (3?/  G  S)[x  - 

-^y  A  y  e  A]}. 

(13) 

440  T.  Moor  and  J,  M.  Davoren 


For  formulas  v?  6  >C(Rel,  Prp)  and  models  2H  of  signature  (Rel,  Prp),  we  say: 
is  satisfied  at  state  s  in  971,  written  Wl,s\=  ip,  if  s  €  and  (p  is  true  in 

9H,  or  QJl  satisfies  (p,  written  9}Z  N  if  \<p\^  =  S. 

In  encoding  the  control  problem  and  input  data  in  modal  logic,  we  work  in 
an  LTS  model  ^)Jlo  over  the  plant  state  space  S  :=  X  CW^.  The  set  of  atomic 
proposition  symbols  is  Prpo  =  {Bad}  U  {Ef,  \  k  e  K},  with  the  self-evident 
denotation  sets.  The  alphabet  Relo  of  relation  symbols  will  grow  dynamically 
in  the  course  of  the  synthesis  algorithm  (but  will  still  be  finite,  due  to  finite 
termination).  The  relation  symbols  divide  into  four  sorts,  which  we  indicate  by 
consistently  using  the  same  letters,  adorned  with  subscripts  and  superscripts 
when  needed.  We  will  have  relation  symbols  e  for  evolution  relations  and  f  for 
flow  (or  orbit)  relations^  symbols  r  for  reset  relations]  and  symbols  S  for  metric 
tolerance  relations. 

Definition  5.  Given  a  flow  ^  :  X  x  — >  X  (possibly  a  partial  function)  and 

any  set  A  Q  X  C  R^,  define  a  relation  e{A,^)  G  X  x  X  of  evolution  along  ^ 
restricted  within  A,  by: 

X  a:'  (3t  eR'^)[x' =  ^(x,t)  A  (Vs  G  [0,t])  d5(a:,  s)  G  A  ] .  (14) 

The  unrestricted  orbit  relation  /(^)  C  XxX  is  the  special  case:  /(^)  =  e(X,^). 

This  precisely  captures  the  notion  of  a  hybrid  trajectory  segment,  taking 
A  =  Invq  and  ^  for  each  control  mode  q  ^  Q.  For  =  e(A,^),  a 

formula  (0)99  denotes  the  subset  of  states  in  A  from  which  there  is  a  curve  along 
that  reaches  some  99-state,  and  stays  within  A  at  all  intermediate  points;  this 
is  the  standard  notion  of  backwards  reachability  extensively  used  in  the  hybrid 
systems  literature.  The  dual  [e]  operator  expresses  invariance,  since  [e]^?  denotes 
the  set  of  points  all  of  whose  e-successors  are  99-states.  The  compound  AA[e](e)99 
denotes  the  set  of  states  in  A  all  of  whose  e-successors  have  a  further  e- successor 
which  satisfies  99,  and  so  captures  the  notion  of  inevitably  reaching  a  99-state. 
This  compound  construct  is  an  essential  ingredient  of  our  synthesis  algorithm, 
where  in  addressing  the  event  sequence  requirement  (S2),  we  need  to  identify 
states  that  are  inevitably  driven  to  certain  local  goal  regions.  Figure  2  illustrates 
the  difference  between  the  inevitability  formula  A  A  [e](e)G  and  the  backwards 
reachability  formula  {e)G,  where  G  denotes  a  local  goal. 

The  reset  relations  under  study  are  elementary,  so  (rg,g/)^°  =  test.Grdq^qr . 
In  this  case,  the  modal  operators  (r^^g/)  and  [rg^g/]  can  be  eliminated: 

(rg,g/)99  o  (Grdg^g.  A99)  and  [rg,g']99  (Grdg,g/ 99)  .  (15) 

For  metric  tolerance  relations  5^^  =  Bs,  a,  formula  {S)(p  denotes  the 
expansion  of  the  set  of  99-states,  since  Bs{A)  =  Pre^(B5)(A).  The  dual  box 
formula  [5](p  denotes  the  5 -contraction  of  the  set  of  99-states,  meaning  the  set  of 
points  in  [99]^°  around  which  one  can  fit  a  5-ball  wholly  inside  [9?]^°. 

An  axiomatic  Hilbert-style  proof  system  capturing  basic  properties  of  the 
modal  operators  of  evolution,  flow  and  metric  tolerance  relations  is  given  in 
[6],  Section  5.  These  axioms  also  may  form  a  basis  for  employing  automated 
reasoning  tools,  e.g.  tableaux  proof  systems  [9]. 
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Fig.  2.  Denotation  of  inevitability  and  backwards  reachability  formulas 


5  Abstract  Algorithm  of  Synthesis  Procedure 

Our  solution  to  the  control  problem  consists  of  two  parts.  First,  we  strategically 
construct  a  finite  number  of  subsets  of  X,  defined  in  terms  of  the  input  data  Fc, 
Bad,  Ekj  next  and  5.  Formally,  this  construction  is  given  as  an  abstract  algorithm 
where  the  sets  of  states  are  defined  by  modal  logic  formulas.  The  algorithm  is  a 
fine-tuned  variation  of  the  one  presented  in  [6].  In  particular,  the  proof  of  finite 
termination  as  given  in  [6]  carries  over  without  change.  The  algorithm  may  either 
terminate  with  failure  or  indicating  success.  In  the  former  case  it  produces  some 
diagnostic  output,  as  described  below.  In  the  case  of  successful  termination, 
the  second  part  of  our  solution  procedure  uses  the  constructed  sets  of  states  to 
assemble  our  nominal  closed-loop  hybrid  automaton  H  and  the  set  Good.  The 
pair  {H,  Good)  then  is  guaranteed  to  fulfill  the  performance  specifications  (Sl)- 
(S4).  It  is  in  this  second  part  that  the  present  work  departs  essentially  from  [6] 
and  extends  the  scope  of  our  method  to  the  plant  parameter  variation  class 

The  first  part  of  our  procedure  is  given  in  Algorithm  1;  see  [6)  for  a  more 
detailed  exposition  including  graphical  output  for  a  nontrivial  example.  Given 
the  page  constraints  on  this  short  paper,  we  are  restricted  to  a  brief  discussion 
of  the  individual  steps  of  the  algorithm.  We  begin  by  taking  the  given  metric 
parameter  S  and  decomposing  it  as  a  sum  5  =  2<5i  +  252,  with  5i  >  62  >  0. 
Roughly  speaking,  is  used  as  “wiggle  room”  in  order  to  cope  with  parameter 
variations  in  the  vector  fields,  while  62  gives  some  extra  allowance  required  for 
an  implementation  based  on  approximated  evaluation  of  the  modal  operators. 
In  the  initialisation  phase,  the  formula  Danger denotes  the  states  that  are 
dangerous  from  the  viewpoint  of  the  block  Eki  the  outright  Bad  states  and  the 
relative  bad  states  in  blocks  Ek'  with  k'  not  next-ielated  to  k.  The  formula 
denotes  Ak,  the  initial  (^-expansion  of  Ek.  The  formula  Goal|.^o  denotes 
the  states  that  are  well  inside  Ek',  in  the  {5  —  52)-contraction  of  Ek',  for  some 
k'  €  next{k). 

The  main  routine  consists  of  an  outer  j -iter  at  ion  which  runs  the  core  routine 
for  successive  j  and  each  k  e  K.  The  purpose  of  the  core  routine  is  to  identify 
states  in  safely  driven  into  the  Goal^-^o-  The  iteration  in  i  is 

with  respect  to  the  number  of  control  switches  required  to  achieve  this  local 
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Algorithm  1  Abstract  algorithm  for  computing  sets  for  synthesis  procedure 
1;  %  INITIALISATION  % 

2:  j  :=  0  and  z  :=  0 

3:  FOR  ALL  k  e  K  DO 


4:  Danger^  (<5i)Bad  V 


5:  Goal<“) 


^next{k)\j{k) 


A<“'  (i)E, 


'  k' ^next{k) 


[i5  -  i52l  E 


•k' 


Drop'“> 


Goal 


(0) 

‘fc.O 


6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 

17 

18 
19: 
20: 
21 
22 

23 

24 

25 

26 
27 


%  MAIN-ROUTINE  % 

REPEAT  %  FOR  j  =  0, 1, . . .  % 

%  CORE-ROUTINE(fc,  j)  iox  k  e  K  % 

FOR  ALL  k€KDO 
REPEAT  %  FOR  i  =  0, 1, . . .  % 
FOR  ALL  c  6  C  DO 


^  Ai^)  A 


,(i)  1 


iDanger, 


Success 

«,2,C 


4|/  j^U) 


A  [e^l J (««',,) (AW>Goal‘^;>)  A  <f4-.(5i>AO> 


0) 


=  Sure«)_, 

def 


A  Success^-^J  ^ 


0) 

k,i  ''  L*^fc,t,cJ 

Goal'-';)^,  =  GoalW  v(Veec  [2^1 +«2]  Finest 

A«],.  &  A<.'''jA^[Mi+i2]Goal<-’>^j 


0)' 


[i2]  (Goal»>  A 


’Goal«>_i) 


^fc,i+l  —  -^k, 

i  :=  i  +  1 
UNTIL  97to  1= 
last{k^j)  i  ~  I 

%  j-th  ATTEMPT  AT  next  COMPATIBILITY  % 
FOR  ALL  keK  DO 

y  f  aW 


Pick<-''> 


Drop 


(i+l)  def 


k'  ^next(k) 
0) 


Drop 


a(j  +  1)  def  *  (0) 

j  :=  j  d- 1  and  z  :=  0 

UNTIL  9Jlo  N  A 


(AL';^AGoal«^.(,,_.,) 

(<52)(Drop^'^^  A  ^Pick 
0+1) 


.0)' 


keK 


Goali^J^^  Drop, 
(Drop^^~^^  ^  Pick^-"-^^) 


28 

29 

30 

31 

32 

33 

34 

35 

36 


%  FINAL  CLEAN-UP  % 


IF  [Drop^ 


O-Uifmo  _ 


0  for  some  k  G  K  THEN 


terminate  &:  report  incompatibility  between  k  and  its  nei^^-successors 

ELSE 

final  j  ~  I 

FOR  ALL  k  G  K  and  z  G  /fc  — ^  {0,  •  •  •  ,  last(k,  final)}  and  c  G  C  DO 
Ak,i  fe'  A[{r°'>  (efc,i,e)”“ 

GoaU.i  Goalifr“'>  Fine^.i,  " 
terminate  with  success 


,e  ='  Fine^-^7'> 
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goal.  In  the  iteration,  Goal^^]  accumulates  the  states  that  can  be  driven  to  the 
initial  Goal^-^j,  by  at  most  /  switches,  while  denotes  states  which  have 

been  not  resolved  so  far.  The  formula  ^  identifies  the  states  in  which 

can  be  driven  to  Goal^^j  using  control  c,  and  done  so  safely  by  being  kept  out 
of  Danger^.  Note  that  the  recursive  definition  of  in  line  17  involves  the 

terms  A^^)  and  Goal^^j  within  the  scope  of  an  odd  number  of  negations.  Thus  it 
cannot  be  coded  as  a  /i-calculus  formula,  and  in  particular  the  inner  i-iteration 
is  essentially  different  from  fixed  point  iterations  of  maximal  invariant  sets  as 
used  in  game-theoretic  approaches  to  safety  problems  for  hybrid  systems  [19]. 

While  the  core  routine  works  on  solving  the  problem  locally,  within  the  in¬ 
dividual  Ak,  the  outer  j-loop  checks  that  these  local  solutions  can  be  merged 
to  form  a  global  controller.  Within  each  Ak,  the  region  where  the  local  solu¬ 
tion  finally  “drops-off”  states  is  denoted  by  Dropip.  The  region  where  such 
states  can  be  “picked-up”  by  adjacent  local  solutions  is  identified  by  Pick^-^^;  for 
compatibility  between  local  solutions,  Pick^^^  is  required  to  contain  Brop)^\ 
If  this  is  not  the  case,  local  goals  are  suitably  reduced. 

Suppose  Algorithm  1  terminates  with  success.  Then,  the  nominal  closed-loop 
system  H  and  initial  states  Good  are  defined  as: 

•  Q  {  (fc,z,c)  G  A  X  N  X  C  I  i  G  4  and  ^  0  } 

•  ’=  Fc  for  all  (/c,  i,c)  eQ 

•  for  each  q  =  (/c,  z,  c)  G  Q,  set 

InVq  :=  |[((5i)Afc,i]^°  n  mt(|[-<(<5i)(AA:,i  A  Goalfc,i)]^°) 

•  E  :=  {((k,i,c),{k',i\c'))  eQ  xQ  \k' Gnext{k)  or  {k'  =  k  and  i' <  i)} 

•  for  each  {q^q')  =  ((A:,  z,  c),  (A:',  i',  c'))  G  E  set 

Grdq^q^  :=  InVq  H  [(25i)(Afc,z  A  Goa\k,i)  A  FineA:^^^c'l^° 

•  test.Grdq^q>. 

•  Good  =  n  |[Finefc,i,cl®^°) 

6  Correctness  and  Robustness  of  Synthesis  Procedure 

Theorem  1.  Let  \  X  ,  c  £  G ,  v  €  V  (Z  he  a  finite  family  of 

parameter  dependent  vector  fields,  where  the  nominal  case  is  denoted  by  Fc  = 
c  £  G.  For  given  specification  data  Bad  C  X,  {Ek}kGK,  next  C  K  x  K  and 
S  >0,  subject  to  assumptions  (A0)-(A5),  suppose  Algorithm  1  terminates  with 
success,  and  H  is  the  nominal  closed-loop  hybrid  automaton  as  above.  Then  there 
exists  a  parameter  bound  s  >  0  such  that  for  every  in  the  variation  class  LF , 
the  pair  {H^ ,Good)  satisfies  each  of  the  performance  specifications  (S1)-(S4). 

The  proof  of  Theorem  1  follows  the  same  general  line  of  argumentation  as  in 
[6].  In  this  outline,  we  focus  on  the  extra  challenges  of  the  variation  class  . 

We  begin  by  choosing  a  variation  bound  e  >  0  such  that  perturbed  integral 
curves  must  remain  within  a  5i-tube  around  the  nominal  curve.  This  is  done  by 
applying  Proposition  1  for  each  q  =  {k,  i,  c)  G  Q,  with  D  =  c/(|Fineg  Alnv^p®) 
and  a  metric  tolerance  of  ~5i.  From  the  construction  of  Successg  (see  Alg.  1, 


444 


T.  Moor  and  J.  M.  Davoren 


line  14)  we  conclude  that  any  nominal  curve  starting  in  Fine^  leaves  {(5i) 
via  A  Goalfc^e-  Then,  by  the  definition  of  InVg^  each  nominal  curve  starting 
in  D  leaves  Bs^  {D),  The  requirements  of  Proposition  1  are  fulfilled  and  we  get  a 
variation  bound  £{q)  >  0  dependent  on  q.  We  choose  e  :=  min{E(g)  |  g  €  Q}  as 
a  witness  of  the  bound  claimed  by  Theorem  1.  In  what  follows,  fix  an  arbitrary 

The  high  level  strategy  is  to  identify  a  list  of  modal  logic  formulas  whose 
truth  in  provides  sufficient  conditions  for  the  specifications  (S1)-(S4)  to  be 
satisfied  by  any  in  The  crucial  modal  formulas  (T1)-(T4)  are  analogs 
of  those  in  [6],  and  are  required  for  each  q  =  (A:,z,  c)  G  Q,  {q,  q')  G  E  and  for  the 


perturbed  flow  relations  :=  e{lnvq, 

for  q  e  Q. 

(Tl) 

( InVq  A  Fineg  ) 

K1  (-^i)  Fine, 

(T2) 

( Invg  A  Fineg ) 

[test.Grdg^g/]  Fineg/ 

(T3) 

( InVg  A  Fincg  ) 

Kl(ep(V,-eB(,)  Grd,.,.) 

(T4) 

( InVg  A  Finog  ) 

— )■ 

(fo)-Inv, 

In  [6],  the  corresponding  formulas  are  derived  directly  from  the  statement  of 
the  algorithm  together  with  the  explicit  assumptions.  Here,  we  are  proving  cor¬ 
rectness  of  a  variant  and  therefore  need  to  exploit  the  relationship  between 
the  perturbed  modal  operators  and  their  nominal  counterparts  in  which  the 
algorithm  is  formalised;  that  is,  (ej)  and  its  relationship  to  (e^). 

From  Proposition  1  and  our  choice  of  e,  we  can  derive  the  following  relational 
inclusion: 


test.invq  o  test.Fineq  o  ej  C  o  ,  (16) 

where  o  is  relational  composition,  which  we  write  in  left-to-right  word  order. 
This  in  turn  implies  the  truth  in  QJlo  of  the  formula: 

(Inv,  AFine,  A[e,][5i]v?)  KJsp  (17) 

for  any  (p  G  >C(Relo,  Prpg).  Then,  (Tl)  can  be  deduced  from  Fine^  — >  [egjFine^ 
(see  [6],  Lemma  7.2)  together  with  formula  (17),  while  (T2)  is  an  immediate 
consequence  of  the  definitions.  Formulas  (Tl)  and  (T2)  are  used  to  establish 
the  safety  specification. 

From  Proposition  1  and  assumption  (AO),  we  can  derive  the  more  sophisti¬ 
cated  modal  fact: 

( Inv,  A  Fine, )  ( [e^]  (e^  (2Ji)  Goal,  A  (f“)  -,Inv, ) .  (18) 

Formula  (18)  expresses  the  essential  properties  of  the  construct  Success^  (Alg.  1, 
fine  14)  but  now  referring  to  the  perturbed  relations  ej  rather  than  the  nominal 
Eq.  In  particular,  we  use  (18)  to  deduce  (T3),  and  (T4)  is  an  immediate  con¬ 
sequence.  Formulas  (T3)  and  (T4)  are  used  to  verify  step-infinite  liveness  and 
the  event  sequence  specification. 
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Having  deduced  the  modal  conditions  (T1)-(T4),  from  this  point  on,  we 
can  largely  mimic  the  proof  in  [6]  to  establish  that  and  Good  satisfy  each  of 
the  specifications  (S1)-(S4). 

7  Discussion  and  Conclusion 

This  paper  addresses  a  basic  hybrid  control  problem,  namely  the  design  of 
a  switching  control  mechanism  via  guard  and  invariant  sets.  We  use  a  novel 
methodology  based  on  modal  logic  to  solve  this  problem  for  a  significant  list 
of  performance  specifications,  and  we  do  so  in  a  manner  that  is  robust  w.r.t. 
parameter  uncertainty  in  the  differential  equations, 

A  significant  issue  to  be  investigated  in  future  work  is  the  question  of  com¬ 
pleteness  of  the  algorithm;  i.e.  whether  there  exists  a  parameterised  plant  and 
specification  data  such  that  there  is  a  robust  solution  to  the  control  problem 
but  the  algorithm  terminates  with  failure  due  to  next  incompatibility.  In  gen¬ 
eral  one  may  expect  such  incompleteness  to  occur.  So  the  question  arises  as  to 
what  additional  conditions  on  the  input  data  could  ensure  completeness.  A  full 
treatment  of  this  issue  necessitates  the  development  of  more  mathematical  tools 
for  analysing  the  space  of  all  possible  solutions  to  our  control  problem,  leading 
to  an  appropriate  notion  of  switching  controllability. 

As  discussed  in  the  introduction,  our  synthesis  algorithm  can  be  implemented 
on  any  available  model  checking  tool.  There  are  two  main  approaches:  exact 
symbolic  computation,  representing  sets  of  states  by  first-order  logic  formulas 
(e.g.  [1,2,14,15]),  and  approximated  representation,  whereby  sets  are  under-  or 
over- approximated  as  finite  unions  of  cells  (e.g.  [4,13,16]).  We  have  developed 
a  prototype  software  implementation  of  our  synthesis  algorithm  based  on  an 
approximation  using  boxes  generated  by  a  regular  grid,  and  it  is  applicable  to 
arbitrary  linear  differential  equations.  The  software  runs  on  a  massively  parallel 
cluster  effectively  employing  96  CPUs,  and  has  been  tested  on  several  non-trivial 
examples.  This  work  on  approximation  based  model  checking  is  to  be  presented 
in  a  separate  paper. 
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Abstract.  Efficient  algorithms  exist  for  fault  detection  and  isolation  of 
physical  systems  based  on  functional  redundancy.  In  a  qualitative  ap¬ 
proach,  this  redundancy  can  be  captured  by  a  temporal  causal  graph 
(TCG),  a  directed  graph  that  may  include  temporal  information.  How¬ 
ever,  in  a  detailed  continuous  model,  time  constants  may  be  present 
that  are  beyond  the  bandwidth  of  the  data  acquisition  system,  which 
leads  to  incorrect  fault  isolation  because  of  a  difference  in  observed  and 
modeled  behavior.  To  solve  this,  the  modeled  time  constants  can  be 
taken  to  be  infinitely  small,  which  results  in  a  model  with  mixed  con¬ 
tinuous/discrete,  hybrid  behavior  that  is  difficult  to  analyze  because  the 
causality  of  the  directed  graph  may  change.  In  this  paper,  to  avoid  the 
combinatorial  explosion  when  using  a  bank  of  TCGs  in  parallel,  causal 
paths  are  parametrized  by  the  state  of  local  switches.  The  result  is  a  hy¬ 
brid  model  that  produces  parametrized  predictions  that  can  be  efficiently 
matched  against  observed  behavior. 


1  Introduction 

To  reduce  cost,  improve  performance,  and  to  manage  the  complexity  of  large 
engineered  systems,  functional  redundancy  can  be  employed  in  fault  detection 
and  isolation  (FDI).  In  this  approach,  a  system  model  links  measured  variables 
by  their  functional  relations,  facilitating  the  computation  of  redundant  values  for 
selected  system  variables.  In  general,  the  system  model  can  be  of  a  continuous 
or  discrete  nature.  In  case  of  a  continuous  model,  often  parameter  and  state 
estimation  techniques  based  on  a  state  space  model  of  the  system  are  used  for 
FDI  [1,4].  In  case  of  a  discrete  event  approach,  models  that  capture  failure  modes 
and  transition  sequences  are  applied  [5,15,16].  Both  these  methods  have  proven 
themselves  successful  in  their  respective  applications. 

Previous  work  [8,9]  has  focused  on  qualitative  parameter  estimation  of  con¬ 
tinuous  system  models.  These  models  are  represented  by  a  temporal  causal  graph 
(TCG)  that  is  automatically  derived  from  a  bond  graph  model  of  a  physical  sys¬ 
tem  [8,11].  This  work  revealed  the  importance  to  design  the  model  in  harmony 
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with  the  data  acquisition  system,  i.e.,  behavior  that  is  beyond  the  bandwidth  of 
the  data  acquisition  system  should  not  be  included  in  the  model  as  it  leads  to 
incorrect  fault  isolation  [2]. 

Removing  large  and  small  parameters  from  the  system  model  causes  the 
following  model  characteristics  that  complicate  the  FDI  task: 

—  Algebraic  loops  may  emerge.  Because  of  the  passive  behavior  of  physical 
processes,  these  algebraic  loops  have  negative  gain,  and,  therefore,  any  qual¬ 
itative  ±  deviation  is  reversed  when  propagated  around  the  loop.  This,  in 
turn,  leads  to  many  unknown  values  of  system  variables  in  a  qualitative 
sense. 

—  In  case  of  abrupt  faults  that  cause  mode  changes,  higher  index  systems 
may  arise  with  algebraic  constraints  between  time  derivative  behavior.  These 
systems  may  exhibit  impulsive  behavior. 

—  The  direction  of  the  computational  causality  in  the  model  may  change.  When 
abrupt  faults  cause  component  parameter  changes  to  values  that  are  taken 
to  be  infinitely  large  or  small,  they  are  effectively  removed  from  the  model, 
which  changes  the  model  configuration,  and,  in  effect,  the  model  becomes  of 
a  switched  continuous,  hybrid,  nature. 

Other  work  [3,12],  addresses  the  first  two  issues  whereas  this  paper  focuses  on 
the  hybrid  diagnosis  problem. 

In  order  to  deal  with  the  change  of  causality,  the  TCG  can  be  derived  for  each 
possible  system  configuration  or  mode.  However,  in  case  of  many  locally  acting 
switches,  the  combinatorial  explosion  quickly  leads  to  an  intractable  problem. 
These  problems  can  be  mitigated  to  some  extent  by  dynamically  generating  the 
TCG  of  each  possible  system  mode  in  response  to  a  failure.  This  may  still  result 
in  a  problem  with  large  computational  complexity  which  can  be  further  reduced 
by  measuring  system  variables  that  indicate  specifically  which  local  switches 
may  have  occurred  [13]  and  predictions  for  each  of  the  variables  that  determine 
different  causal  assignments  are  required  to  be  made  and  analyzed.  Once  a  set 
of  possible  TCGs  is  available,  Gaussian  decision  techniques  have  been  applied 
to  compute  the  most  likely  mode  of  continuous  behavior  [7]. 

Recent  attention  to  hybrid  diagnosis  [7,14]  concentrates  on  efficiently  pro¬ 
cessing  a  set  of  TCGs.  This  paper  describes  how  a  hybrid  model  can  be  made 
amenable  to  the  diagnosis  algorithms  that  were  developed  in  previous  work  [8,9] 
by  systematically  generating  one  parametrized  TCG.  In  this  graph,  the  directed 
links  are  enabled  by  conditionals  that  correspond  to  the  mode  in  which  these 
links  are  present.  The  result  is  a  set  of  predictions  that  are  parametrized  by 
the  state  of  the  local  switches  and  the  diagnosis  problem  then  becomes  one  of 
constraint  satisfaction  [17].  The  solution  to  this  constraint  satisfaction  problem 
contains  the  possible  parameter  changes  (i.e.,  the  faults)  and  the  effect  on  the 
system  mode  that  this  is  required  to  have. 
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2  Preliminaries 

This  section  reviews  the  qualitative  FDI  approach  developed  in  previous  work  [8, 
9].  Instead  of  a  temporal  causal  graph,  though,  the  model  representation  format 
and  processing  will  be  in  qualitative  matrix  algebra,  which  is  easier  to  represent 
and  to  extend  with  the  required  notions. 

Consider  the  one-tank  hydraulic  system  in  Fig.  1.  The  functional  relation 
for  flow,  /fi,  through  the  outflow  pipe  is  given  by  /j?  =  where  pR  is  the 
pressure  drop  across  the  pipe  and  R  is  the  pipe  resistance  to  flow.  The  pressure 
Pr  depends  on  the  pressure  at  the  bottom  of  the  tank,  pc,  according  to  pr  =  pc 
(i.e.,  the  ambient  pressure  is  assumed  to  be  0).  The  rate  of  change  in  the  pressure. 
Pc,  at  the  bottom  of  the  tank  is  given  hy  pc  =  where  fc  ~  fin  —  Ir  and 
fin  is  the  flow  into  the  tank  and  C  is  the  tank  capacity. 


To  derive  qualitative  predictions,  the  system  is  written  as  a  directed  graph 
that  captures  the  causal  (directed)  relations  between  system  variables.  For  the 
one-tank  system,  the  preferred  (integral)  causality  model  description  is 
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where  A  represents  the  time  differentiation  operator  and  A  ^  indicates  integra¬ 
tion  over  time.  The  corresponding  temporal  causal  graph  (TCG)  is  given  in 
Fig,  2, 

The  TCG  can  be  represented  by  a  weighted  adjacency  matrix  where  the 
columns  are  cause  and  rows  are  the  effect  variables  and  the  entries  capture  the 
parameters  on  the  graph  edges.  This  is  called  the  temporal  causal  matrix  (TCM), 
that  is 

'l 0  0 

0  1-10 
0  0  1  R-^ 

10  0  1 


PC 

fc 

fn 

PRJ 


(2) 


for  the  TCG  in  Fig.  2. 


450  P.J.  Mosterman 


Pc 

c-’x-'t 


^  fr 


Pr 

|r*' 


Fig.  2.  TCG  of  the  one-tank  system. 


Our  diagnosis  engine  TRANSCEND  [6]  relies  on  qualitative  information  to 
achieve  diagnosis.  In  this  framework,  only  the  three  values  0,  -f  are  used  to 
indicate  values  that  are  too  low,  normal,  and  too  high,  with  respect  to  some 
nominal  value,  respectively.  For  example,  a  value  of  a  model  variable  that  is 
measured  to  be  above  its  nominal  value  is  marked  +.  In  case  the  outflow  of  the 
tank  system  in  Fig.  1  is  too  high,  this  is  represented  by  /^. 

Note  that  in  a  qualitative  representation,  the  parameters  R  and  C  correspond 
to  direct  relations  between  variables,  and,  therefore,  they  can  be  replaced  by 
value  1.  This  results  in  a  qualitative  system  where  1  and  —1  represent  direct  and 
reverse  relations,  respectively. 

To  find  parameter  deviations,  in  previous  work  a  backpropagation  algorithm 
is  used.  In  qualitative  matrix  algebra  this  is  equivalent  to  repeated  multiplication 
of  the  initial  deviation  with  the  transpose  TCM,  Here,  for  this  results  in  the 
sequence  of  vectors 
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The  parameters  R  ^  and  C~^  are  fault  hypotheses  and  replaced  by  1  after 
they  are  generated  because  R  and  C  are  positive  parameters,  and,  therefore, 
in  a  qualitative  framework  they  represent  direct  relations.  Also,  qualitatively 
1  —  1  is  unknown,  Once  all  variables  are  unknown,  no  further  parameter 
deviations  can  be  hypothesized  (the  remaining  candidates  that  are  not  generated 
in  Eq.  (3)  are  —R  ^  and  —C  ^),  The  resulting  set  of  possible  faults  is,  therefore, 
R~^  or  C~'^  too  high,  i.e.,  (the  remaining  candidates  are  {i?+,C'+}). 

Physically,  these  fault  candidates  correspond  to,  e.g.,  leakage  in  the  outflow  pipe 
{R~)  or  an  object  that  has  fallen  into  the  tank  (C“). 

Next,  predictions  of  future  system  behavior  are  generated  for  each  of  the  pos¬ 
sible  parameter  deviations,  R~  and  C~ .  From  the  TCM,  their  initial  deviations 
are  found  to  be 


R- 

’0" 

0 

1 

,  C  — )• 

0 

0 

0 

0 

(4) 


To  achieve  a  suffiently  high  order  prediction  for  the  measured  variable,  the 
initial  deviation  is  repeatedly  multiplied  with  the  TCM.  Here,  a  second  order 
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and  can  be  used  for  efficiently  generating  predictions  for  other  fault  candidates. 

The  polynomials  in  A  are  equal  to  the  qualitative  signatures  generated  in 
previous  work  [8,9].  For  this  example,  the  signature  for  the  measured  variable  is 
where  the  superscripts  indicate  the  qualitative  values  of  the  time  deriva¬ 
tive  behavior  with  increasing  order  from  left  to  right,  i.e.,  there  is  a  positive 
discontinuous  change  with  negative  slope  that  increases.  For  the  pressure  at 
the  bottom  of  the  tank,  the  prediction  is  i.e.,  no  discontinuous  change  in 

pressure  occurs  and  the  pressure  is  decreasing. 

This  method  works  well  if  the  system  of  equations  that  describes  continuous 
behavior  is  fixed.  However,  in  case  discrete  switches  cause  changes  in  the  continu¬ 
ous  model,  signatures  for  each  mode  have  to  be  generated.  This  quickly  becomes 
intractable,  and,  therefore,  for  these  system  models  a  parametrized  formulation 
is  advantageous. 


3  Hybrid  Models  for  FDI 

For  the  qualitative  FDI  approach  to  be  effective,  it  is  imperative  that  the  modeled 
time  constants  are  observable,  i.e.,  within  the  bandwidth  of  the  data  acquisition 
system.  If  a  parameter  that  models  an  abrupt  fault  changes  to  a  very  large  or 
small  value,  it  may  correspond  to  a  time  constant  that  cannot  be  observed,  and, 
therefore,  this  behavior  needs  to  be  abstracted  from  the  model.  This  causes  the 
model  to  be  of  a  switched  continuous,  hybrid  nature. 

In  general,  modeled  discontinuities  result  in  causal  changes.  Therefore,  the 
TCM  may  take  several  different  forms  and  so  do  the  corresponding  predictions 
of  future  behavior,  depending  on  whether  a  mode  change  occurs.  Consider  for 
example  a  valve  that  controls  the  outflow  in  Fig.  1  in  a  binary  manner,  i.e.,  either 
there  is  an  outflow  determined  by  the  Bernoulli  resistance  (ai  =  1)  or  there  is 
no  outflow  (cKi  =  0).  When  the  switch  is  modeled  as  a  discontinuous  change, 
the  corresponding  model  includes  a  change  in  causality  when  the  control  valve 
switches  its  state.  If  it  is  open,  the  pressure  pc  determines  the  outflow  Jr  and 
if  it  is  closed,  Jr  ~  0,  which  determines  the  pressure  drop  across  the  pipe  to 
be  PR  —  /rR  =  0.  To  handle  the  change  in  TCM,  the  causal  relations  can  be 
parametrized  to  make  them  dependent  on  the  mode  of  operation. 
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To  this  end,  first  the  system  is  described  in  a  noncausal  form  by  using  implicit 
equations.  An  implicit  model  of  the  one  tank  consists  of  the  following  equations 


0  =  Cpc  -  fc  (7) 

0  ~  fc  —  fin  +  Ir  (8) 

0  =  R/r  -  PR  (9) 

0  =  ai(pR  -  pc)  +  (1  -  ai)fR  (10) 


From  Eq.  (10),  in  case  the  control  valve  is  open,  ai  =  1,  and  PR—pc,  when  the 
control  valve  is  closed,  ai  =  0,  and  fR  =  0. 

The  TCM  for  this  system  of  equations  contains  the  relations  between  each 
of  the  variables.  For  example,  Eq.  (7)  embodies  a  temporal  relation  between  pc 
and  fc  and  Eq.  (10)  a  direct  relation  between  pc  and  pr  that  is  only  active 
when  ai  ^  0.  The  TCM  then  becomes 


1  0  ai  I  [pc 

XC  1  -10  fc 

0  -1  1  /fi 

oi  0  jR  1  Lpk 


(11) 


and  causal  links  from  pc  to  pr  and  from  pR  to  pc  are  only  active  when  the  system 
is  in  mode  ai,  A  special  case  arises  for  ai  =  0  which  implies  fR  =  0.  This  efltect  is 
not  present  in  the  TCM  because  it  is  not  a  relation  between  variables.  However, 
it  contains  essential  diagnostic  information  about  system  behavior  that  can  be 
included  by  an  input  vector 

(12) 


where  the  —  sign  is  because  the  flow,  /r^  is  positive  during  normal  operation, 
and,  therefore,  its  deviation  is  —  when  the  valve  closes  (possibly  inadvertently). 

Diagnosis  now  proceeds  to  predict  future  behavior,  i//,  for  each  hypothesized 
fault,  /,  and  both  possible  configurations  (ai  ==  0  and  ai  =  1).  To  this  end,  the 
TCM,  A,  raised  to  a  sufficiently  high  power,  n,  operates  on  the  sum  of  the  input 
vector,  u,  and  each  of  the  initial  deviations,  df,  generated  from  the  hypothesized 
faults, 

Vf  =  A^{df  +  u)  (13) 

These  predictions  are  then  compared  against  actual  observations  to  prune  the 
fault  hypotheses  and  find  the  correct  fault. 

Note  that,  to  facilitate  a  qualitative  algebra,  the  (1  —  o;)  construct  with 
a  e  {0, 1}  cannot  be  used  to  (de) activate  relations  because  in  a  qualitative  sense 
(1  ~  is  unknown  instead  of  0.  Therefore,  -la  is  used  to  indicate  a  quantitative 
evaluation  of  (1  —  a)  so  that  —<a  produces  a  value  {0, 1}. 

For  the  initial  deviation  that  corresponds  to  R~  in  Eq.  (4)  and  the  input 
vector  in  Eq.  (12),  after  multiplying  with  the  TCM  five  times,  the  prediction 
becomes 


(1  --nQi) 


(14) 
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Compared  with  the  prediction  derived  from  the  explicit  system  in  Section  2  this 
shows  impulsive  behavior  because  of  the  positive  powers  of  A  and  other  spurious 
behavior  because  all  possible  relations  are  present  in  the  TCM.  In  other  words, 
for  a  given  causal  assignment  all  other  relations  are  present  as  well  even  though 
these  may  not  be  consistent  with  the  given  causal  assignment. 

To  demonstrate  that  such  an  extensive  set  of  relations  quickly  leads  to  con¬ 
tradiction,  consider  an  implicit  relation  0  =  Xi X2 xs  with  TCM 


'xi ' 

. 

.^3. 

(15) 


Because  in  a  qualitative  sense  1-1  is  unknown,  this  leads  to  unknown  predictions 
as  soon  as  the  TCM  is  raised  to  a  power  >  1  (e.g.,  x^  x^  ^ 

and  xi  is  unknown).  This  problem  can  be  circumvented  by  committing  to  one 
causal  assignment  only.  In  matrix  form,  this  is  achieved  by  using  binary  selection 
variables,  ki  G  {0, 1}, 


1  —kik2  —kik2 

—  ki-ik2  1  —ki-~ik2 

—  k2->ki  —k2-'k\  1 


(16) 


and  the  matrix  is  invariant  under  multiplication. 

In  summary,  to  design  an  approach  for  diagnosis  based  on  hybrid  models,  the 
TCM  is  derived  from  an  implicit  model  formulation  that  includes  mode  selection 
parameters,  to  switch  between  equations.  The  possible  causal  assignments  of 
ternary  and  higher  relations  are  then  made  mutually  exclusive  by  introducing 
selection  parameters,  ki.  If  possible,  the  parameters  can  be  related  to  ki  and 
the  TCM  contains  only  mode  selection  parameters,  and,  therefore,  produces 
fault  hypotheses  and  predictions  that  are  parametrized  by  a*  only. 


4  A  Case  Study 

To  make  the  implicit  approach  suitable  for  diagnosis,  it  must  deal  with  additional 
causal  paths  and  the  possible  conflicts.  Consider  the  two  tank  system  in  Fig.  3 
with  externally  controlled  outflow  valves  on  the  left  and  right  and  a  pressure 
controlled  valve  between  the  left  and  right  tank.  An  implicit  quantitative  model 
of  this  system  could  look  like 


0  —  fin  “b  fCi  “b  fRi,i  "b  fRi2 

0  =  ai(-pci  +PRi2  +PC2)  -b  (1  -  «l)/i?i2 
0  =  a2{pci  -  PRti  )  +  (1  -  «2)/i?bi 

0  =  QsCPCa  ”  PRb2)  +  (1  -  0^3)/Rb2 
0  =  fc2  +  fRb2  ~~  fRi2 
0  =  Cipci  -  fci 
0  =  C2PC2  -  fC2 
0  =P/25i  -RblfRbi 
0  =  PRb2  ~  ^b2fRb2 
0  =  PR12  “  ^12/i?i2 


(17) 
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where  ai  are  mode  selection  parameters  and  ai,  a2,  and  as  correspond  to  the 
state  of  the  middle,  left,  and  right  valves  in  Fig.  3,  respectively,  where  otj  =  0 
implies  the  valve  is  closed  and  a*  =  1  that  the  valve  is  open. 


Fig.  3.  Two  tanks  with  outflow  valves  and  a  pressure  controlled  connecting  valve. 


This  model  contains  a  number  of  ternary  relations  (input  variables  are  not 
considered  as  fault  candidates)  and  when  a  deviation  is  propagated,  multiple 
possible  paths  are  taken.  To  prevent  this,  the  paths  can  be  parametrized  as 
demonstrated  in  Section  3  (the  binary  relations  are  mutually  consistent), 
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For  this  model,  the  causality  of  some  of  the  binary  relations  is  fixed  for  each 
possible  mode  and  incorporating  this  a  priori  knowledge  leads  to  a  more  con¬ 
strained  model.  For  example,  the  relation  0  =  a2(pci  -PRbx)  i^ads  to  two  entries 
in  the  TCM,  one  for  pc^  -4  and  one  for  ^  pc^ .  Analysis  reveals  that 
the  latter  causal  relation  is  never  used  for  any  configuration  of  valve  states,  and, 
therefore,  the  corresponding  entry  in  the  TCM  can  be  removed.  The  matrix  en¬ 
tries  in  Eq.  (18)  that  vanish  because  of  pre-processing  are  marked  by  a  bounding 
box. 

The  causality  of  the  ternary  relations  can  be  analyzed  exhaustively  because 
it  only  involves  a  limited  number  of  local  constraints.  Causal  analysis  of  the 
system  of  equations  shows  that  although  the  causality  of  the  ternary  equations 
may  change,  the  changed  causality  corresponds  to  the  vanishing  (deactivating) 
of  an  edge.  For  example,  the  causality  of  0  =  ai  (~pci  T P/?i2  +PC2 )  changes  when 
ai  changes  its  value.  But,  for  the  state  -^Q;i,  the  equation  is  not  active  anymore. 
Therefore,  this  need  not  be  explicitly  modeled,  and  the  relation  between  the  a^ 
and  ki  degrades  to  the  fixed  values  =  1,  ^2  =  1,  =  1,  ^4  =  0,  ^5  :=  0,  and 

fce  =  1. 


Diagnosis  of  Physical  Systems  with  Hybrid  Models  455 


Fig.  4.  The  temporal  causal  graph  of  the  two-tank  system. 


In  Fig.  4  the  temporal  graph  of  the  TCM  is  shown  to  clarify  the  relations 
between  system  variables.  The  dashed  edges  are  those  that  are  present  in  the 
original  implicit  formulation  because  of  ternary  relations  but  that  are  removed 
based  on  a  mode  dependent  causal  analysis.  The  undirected  edges  are  implicit 
binary  relations  and  can  be  decomposed  into  two  edges  with  opposite  direction 
(corresponding  to  the  two  entries  in  the  TCM)  to  be  compatible  with  the  tem¬ 
poral  causal  graph  format  used  in  previous  work  [8,9] .  Note  that  in  many  cases, 
graph  propagation  is  more  efficient  than  matrix  multiplication,  especially  in  case 
of  sparse  matrices. 

After  replacing  the  parameters  with  their  qualitative  equivalent,  the  resulting 
TCM  is  given  by 
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where  the  boxed  entries  are  those  that  correspond  to  bidirectional,  non-causal, 
edges  (in  this  particular  case,  these  could  still  be  made  mode-dependent,  where 
the  entries  above  the  diagonal  become  ai  and  below  become  -iQj). 

The  predictions  of  the  TCM  are  parametrized  by  the  active  mode.  This  leads 
to  more  efficient  diagnosis  compared  to  the  use  of  a  bank  of  TCMs,  which,  in  this 
case  of  three  switches,  would  consist  of  eight  TCMs  that  need  to  be  processed 
separately.  For  example,  in  case  of  a  measurement  R^2 
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hypotheses  that  results  in  the  prediction 

~ai  4-  Qi 

— aia2A~^ 

Qi A“^  —  ai A”^ 

—  1  +  ckiA  ^  "1-  Q^sA  ^  —  ctiA  ^ 

1  —  cksA”^  +  aaA"^ 

—  Q:i  A~^ 
ai A“^  —  aiA“^ 

-A“^  +  a3X~^ 

—  aia2X~'^ 

1  —  c^sA  ^  +  cksA  ^ 

In  addition,  the  input  vectors  for  ^ai,  -ia2  and  -las  are  determined  to  be 

■0*1  r°i  r^' 

0-10 
-10  0 
0  0  0 

-nai  -)►  Q  ,  ^a2  ^  0  ’  “*<^3  “>  V  ’ 

0  0  0 

0  0  0 

0  0  0 

.  0  J  L  0  J  L  0  _ 

and  their  effect  is  propagated  as  well.  For  -lai,  this  leads  to  the  prediction  for  pci 
to  be  -lai A“^  —  -^aia2X~^  —  or  -iaiA“^  —  ^aia2A~^.  The  combined 

prediction  for  pci  becomes 

-iq;iA“^  -  -<aia2A“^  —  ckiA”^  (22) 

The  parametrized  predictions  can  be  matched  against  further  measurements 
(e.g.,  where  the  second  order  derivative  is  not  measured).  In  case  Q!i,  i.e., 
the  pressure  controlled  connecting  valve  remains  open,  the  prediction  for  pci  is 
— A“^,  a  falling  level  of  liquid  in  Ci  with  second  order  behavior.  This  is  incon¬ 
sistent  with  the  observation  and  the  fault  i2^[ai]  is  rejected  as  a  possible 
explanation  of  the  anomalous  system  behavior.  If  the  new  pressure  in  C2  causes 
the  connecting  valve  to  close,  the  predicted  behavior  of  pCi  changes.  This  can 
be  derived  by  evaluating  the  prediction  with  -nai,  which  yields  A“^  —  a2A“^, 
i.e.,  the  Hquid  level  in  Ci  rises.  In  case  the  left  outflow  valve  remains  open,  a2, 
the  rate  of  increase  decreases  but  if  this  outflow  valve  closes,  the  level  contin¬ 
ues  to  rise.  It  is  easily  verified  that  the  predictions  of  both  fault  hypotheses 
(■^^[^^1^2]  and  are  consistent  with  the  p^  measurement,  and, 

therefore,  possible  causes  of  the  observed  anomalous  behavior.  Further  mea¬ 
surements  are  needed  to  prune  this  set  of  candidates,  as  described  in  detail 
elsewhere  [8,9], 

5  Conclusions 

Algorithms  and  hybrid  models  for  diagnosis  of  physical  systems  are  required  to 
deal  with  configuration  changes  between  modes  of  operation  but  the  combinato¬ 
rial  explosion  prohibits  a  global  enumeration  approach.  This  papers  shows  that 


(21) 
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mode  changes  can  be  modeled  by  locally  activating  and  deactivating  relations 
between  system  variables.  When  relations  are  (de) activated,  the  causal  effect 
between  system  variables  may  change.  This  is  handled  by  including  all  possi¬ 
ble  relations  between  system  variables.  Because  of  the  presence  of  relations  not 
describing  system  behavior  in  a  given  mode,  the  model  may  foster  conflicting 
relations,  which  is  solved  by  introducing  parameters  to  enforce  mutual  exclusion 
between  different  causal  assignments  on  individual  relations.  Performing  local 
analyses  establishes  the  relation  between  these  parameters  and  mode  selection 
parameters.  The  resulting  method  generates  conditional  predictions  that  depend 
on  the  mode  of  the  system  which  allows  for  efficient  execution  of  the  diagnosis 
algorithms. 

The  presented  method  allows  for  a  declarative  prediction  of  future  system 
behavior.  It  has  not  taken  yet  taken  into  account  imperative  mode  switching 
functionality  (e.g.,  a  switching  constraint  such  as  pi  >  P2  causes  a2  =  1).  In¬ 
cluding  this  may  constrain  possible  mode  changes,  and,  therefore,  further  prune 
the  set  of  hypothesized  candidates. 

Note  that  the  analysis  of  interacting  local  switches  is  automated  in  HyBr- 
SlM  [10]  based  on  analysis  of  causal  areas  in  a  bond  graph.  This  forms  the  basis 
for  future  research  into  automatically  performing  the  pre-processing  of  the  re¬ 
lations  between  mode  selection  parameters  and  those  that  ensure  mutual  exclu¬ 
sion  of  different  causal  assignments.  This  should  facilitate  scaling  the  approach, 
because  the  complexity  increases  exponentially  only  with  interacting  switches 
within  one  causal  area.  So,  e.g.,  for  k  causal  areas  with  m  switches,  instead  of 
2^”^  modes,  k2^  modes  have  to  be  analyzed,  and  typically  if  a  hybrid  bond  graph 
modeling  approach  is  useful,  the  number  of  switches  that  interact  directly,  i.e., 
without  dynamic  behavior,  is  low. 
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Abstract.  We  address  systems  which  have  multiple  objectives:  broadly 
speaking,  these  objectives  can  be  thought  of  as  safety  and  performance 
goals.  Guaranteeing  safety  is  our  first  priority,  satisfying  performance 
criteria  our  second.  In  this  paper,  we  compute  the  system’s  safe  operat¬ 
ing  space  and  represent  it  in  closed  form,  and  then,  within  this  space, 
we  compute  solutions  which  optimize  a  given  performance  criterion.  We 
describe  the  methodology  and  illustrate  it  with  two  examples  of  systems 
in  which  safety  is  paramount:  a  two-aircraft  collision  avoidance  scenario 
and  the  flight  management  system  of  a  VSTOL  aircraft.  In  these  ex¬ 
amples,  performance  criteria  are  met  using  mixed-integer  nonlinear  pro¬ 
gramming  (MINLP)  and  nonlinear  programming  (NLP),  respectively. 
Optimized  trajectories  for  both  systems  demonstrate  the  effectiveness  of 
this  methodology  on  systems  whose  safety  is  critical. 


1  Introduction 

Aircraft  collision  avoidance  maneuvers  and  flight  management  systems  are  safety 
critical  systems  for  which  one  would  like  to  guarantee  a  certain  level  of  perfor¬ 
mance:  controllers  for  such  systems  must  address  potentially  conflicting  goals  of 
hierarchical  importance  [1].  The  safety  of  a  system  is  determined  by  its  ability 
to  remain  within  an  allowable  subset  of  the  state  space.  For  example,  in  colli¬ 
sion  avoidance  maneuvers,  the  aircraft  must  remain  separated  by  a  minimum 
distance,  while  in  flight  management  systems,  the  state  of  the  aircraft  must  re¬ 
main  inside  its  aerodynamic  flight  envelope.  Performance  goals  can  be  specified 
in  terms  of  costs  of  deviations  from  desired  routes,  or  in  minimizing  fuel  us¬ 
age.  Combining  controllers  to  meet  these  objectives  is  an  important  and  difficult 
problem:  conflicting  objectives  can  result  in  chattering  and  other  undesirable 
effects  [2].  In  [1],  the  authors  proposed  a  scheme  for  combining  multiobjective 
controllers  for  systems  with  safety  and  performance  objectives.  The  safe  region 
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of  operation  and  the  controller  necessary  to  guarantee  that  the  system  remain 
within  the  safe  region  is  first  determined,  and  the  designer  is  given  the  freedom  to 
choose  a  controller  that  satisfies  performance  constraints  within  this  safe  region. 
This  controller  must  be  overridden  whenever  the  system  reaches  the  boundary  of 
the  safe  space.  In  this  paper,  we  address  the  design  of  the  performance  controller 
under  the  restrictions  of  the  safety  controller.  Our  methodology  combines  the 
Hamilton- Jacobi  approach  of  [3]  (for  systems  in  which  we  can  find  closed-form 
representations  of  the  safe  space)  and  nonlinear  optimization  techniques  [4,5],  by 
viewing  the  restrictions  necessary  for  safety  as  inequality  constraints  in  a  nonlin¬ 
ear  optimization  problem.  A  similar  problem  of  incorporating  state  and  control 
restrictions  has  been  addressed  for  linear  hybrid  systems  with  linear  constraints 
by  using  a  model  predictive  control  framework  [6,7]. 

By  designing  our  controller  in  two  steps,  we  assure  that  the  most  important 
criteria,  safety,  is  always  met,  and  that  the  controller  optimizes  the  performance 
of  the  system  over  the  safe  region  of  operation  for  any  specified  time  horizon. 
This  two-step  process  assumes  that  the  anal3d;ical  solution  for  the  safe  region  is 
known;  however,  in  cases  for  which  there  is  no  analytical  solution,  an  anal5d:ical 
under  approximation  can  be  used.  By  contrast,  a  one-step  method  in  which  safety 
and  performance  are  optimized  in  a  single  cost  function  over  a  fixed  time  horizon 
guarantees  safety  only  over  that  time  horizon  -  although  the  system  will  remain 
outside  of  the  unsafe  set  for  the  time  over  which  performance  is  optimized,  it 
could  potentially  enter  unsafe  set  at  the  next  time-step. 

We  demonstrate  our  method  to  generate  safe,  yet  optimal,  trajectories  on  two 
nonlinear,  safety-critical  systems.  The  collision  avoidance  scenario  involves  the 
lateral  dynamics  of  two  cooperative  aircraft  in  free  flight  [8].  Collision  avoidance 
has  been  an  active  area  of  research  for  contributors  who  have  approached  the 
problem  in  a  variety  of  ways,  including  probabilistic  [9,10],  optimal  [11,12,13], 
and  hybrid  [14,1]  frameworks.  The  focus  of  the  probabilistic  and  hybrid  work  has 
been  on  the  computation  of  safe  operating  regions  for  groups  of  aircraft,  while 
the  focus  of  the  optimal  work  has  been  to  optimize  performance  criteria  over 
a  finite  horizon  while  maintaining  a  5  nmi  radial  separation  between  aircraft. 
The  safety  of  the  latter  solution  depends  on  appropriate  choice  of  time  horizon. 
The  flight  management  system  presented  involves  the  longitudinal  dynamics 
of  a  Vertical  and/or  Short  Take-Off  and  Landing  (VSTOL)  aircraft.  The  safe 
region  of  operation  for  each  mode  of  the  hybrid  system  was  derived  in  [2],  and 
the  stability  of  switched  feedback  linearizing  control  laws  analyzed  in  [15].  The 
naive  combination  of  these  two  controllers  results  in  chattering  and  large  tracking 
errors. 

In  this  paper,  we  compute  optimal  control  laws  which  smoothly  guide  the 
system  through  the  safe  region  of  operation.  Our  methodology  for  multiobjec¬ 
tive  controller  synthesis  involves  three  steps:  analyzing  the  safety  of  the  system, 
representing  the  safe  region  of  operation  in  a  form  suitable  for  a  nonlinear  pro¬ 
gram,  and  then  optimizing  a  desired  performance  goal  constrained  to  lie  within 
the  safe  region  of  operation.  We  demonstrate  our  methodology  for  each  of  the 
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above  steps  with  the  collision  avoidance  and  VSTOL  examples.  We  then  discuss 
our  optimization  results  and  conclude  with  directions  for  further  research. 

2  Problem  Description 

We  address  the  problem  of  combining  safety  and  performance  goals  for  a  hybrid 
system  in  a  single  discrete  mode,  that  is,  for  the  nonlinear  continuous  dynamics: 

x  =  f{x,u)  (1) 

with  state  x  €  X,  and  control  input  u  €U.  Given  an  initial  unsafe  region  G  C  X, 
we  follow  the  method  of  [3]  to  compute  the  maximal  controlled  invariant  set 
contained  in  which  is  denoted  W*  C  X.  W*  represents  those  states  from 
which  there  exists  a  control  input  u  £  U  such  that  the  system  can  remain  in 
W*  for  all  future  time.  We  also  compute  the  set  valued  feedback  control  law 
Usa.fe{^)  which  guarantees  that  the  system  remains  in  W*.  Next,  we  determine  a 
closed-form  representation  for  the  safety  constraints  x  £  W*,  which  we  represent 
as  cw{x)  <  0.  We  then  optimize  the  desired  performance  goal  by  minimizing 
Jperf(2:,w)  over  X  £W*  subject  to  discretized  dynamics. 

Minimize  Jperfi^k^'^k) 

subject  to:  Xk+i  =  fai^k^'i^k)  ^2) 

Xmin  ^  ^  Xmax?  ^min  ^  '^k  ^  f^max 

cwixk)  <  0 

The  control  law  which  results  from  this  optimization  will,  by  construction,  keep 
the  discretized  system  within  the  safe  region  W*  for  the  time  horizon  over  which 
it  is  optimized.  As  with  any  discretization  process,  the  discretized  model  does  dif¬ 
fer  from  the  continuous  model,  allowing  for  unaccounted-for  discrepancies  in  the 
performance  of  these  controllers  on  the  actual  continuous  system.  Discretization 
in  hybrid  systems  is  further  complicated  due  to  the  interaction  of  the  continuous 
dynamics  with  transitions.  In  this  paper  we  use  a  forward  Euler  discretization 
method  and  neglect  any  discrepancies. 


2.1  Collision  Avoidance 

We  consider  the  lateral  dynamics  of  a  two-aircraft  scenario  with  full  coopera¬ 
tion  between  aircraft  (safety  concerns  arise  due  to  finite  control  input).  The  two 
aircraft  travel  at  a  constant  speed  V  in  the  (x,  y)  plane  with  heading  angles  'ipi 
and  'ip2,  respectively.  The  lateral  dynamics  of  the  two  aircraft  are  x  =  /(x,'u), 
where  x  =  [xi  yi  ipi  X2  2/2  ”02]^  ^iid  u  =  {ui  U2]'^ ,  the  roll  angles  of  the  two 
vehicles.  For  i  £  {1,2},  Xi  =  V sin 'ipi,  yi  —  Fcos0i,  ipi  —  ^tanuj,  and 
Ui  £  [-0max,0max],  where  0max  =  27r/9  due  to  allowable  aircraft  roll.  Since 
the  relative  orientation  of  the  aircraft  is  of  main  interest,  we  transform  the 
inertial  two-aircraft  system  into  a  right-handed  relative  frame  of  reference  by 
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defining  the  relative  position  and  heading  of  aircraft  2  with  respect  to  the  in¬ 
ertial  position  and  heading  of  aircraft  1:  ^r),  where  9r  =  —  -02  and 

[xy  y^Y'  =  lO-  —  'ipi)[{x2  —  xi)  (i/2  ~  yi)]^,  R{^)  a  standard  rotation  ma¬ 
trix  through  the  angle  The  relative  dynamics  are  therefore  x^  —  fr(x^^u), 
where 


Xr  =  —V  -{-V  COS  6r  —  tan  Ui 

Vr  —  V  sin  6r  +  ^Xr  tan  ui  (3) 

=  -f^(tanu2  ~  taniti) 

with  x^  =  [xr  Vr  OrY-  The  minimum  aircraft  separation  is  defined  as  5  nautical 
miles.  To  be  safe,  therefore,  the  state  must  remain  in  C,  where 

=  {(xr^Vr^Or)  :  x^  +  >  52}  .  (4) 


2.2  VSTOL  FMS 

Consider  the  longitudinal  axis  dynamics  of  the  VSTOL  aircraft  in  the  TRAN¬ 
SITION  mode,  or  the  mode  in  which  the  thrust  can  be  vectored  from  the  body 
axis  through  90°,  resulting  in  a  wide  range  of  dynamic  behaviors  [2].  The  iner¬ 
tial  coordinates  of  the  aircraft’s  center  of  mass  are  [x^  z)  along  the  horizontal 
and  vertical  axes,  respectively,  and  the  pitch  angle  6  is  the  angle  between  the 
aircraft  body  axis  and  the  inertial  x  axis.  The  flight  path  angle  7,  the  angle  of 
attack  a,  and  the  ground  speed  V  are  defined  as  7  =  tan~^(|),  a  =  ^  -  7,  and 
V  =  Vx^-h  respectively.  The  aerodynamic  equations  for  lift  (L)  and  drag 
(D)  are  given  by  L  =  aLV^(l  ca),  D  ==  aoV^il  +  b{l  -f  ca)^),  with  constants 
b  =  0.02,  c  =  11.42,  gl  =  2.72,  an  =  2.54  determined  from  actual  Harrier  flight 
data  [16]  as  well  as  our  own  estimates.  Further  details  on  the  model  development 
are  available  in  [2].  The  aircraft  nozzles  rotate  from  the  body  axis  through  the 
angle  S  with  rate  S.  We  assume  that  the  autopilot  has  direct  control  over  both 
the  forward  thrust  ui  =  T,  the  pitch  acceleration  U2  =  J9  (through  the  eleva¬ 
tors),  and  the  nozzle  acceleration  =  S.  We  obtain  the  longitudinal  dynamics 
from  the  Newton-Euler  equations 


X 

z 

=  R{9)  Y^{a) 

'~D 

L 

+ 

Ui  cos  (J 
ui  sin  8  —  eu2 

)- 

0 

Mg 

where  e  is  a  small  positive  constant.  The  aircraft  has  mass  M  =  162801b  and  mo¬ 
ment  of  inertia  about  the  pitch  axis  J  =  32000slug-ft2.  Safety  regulations  for  the 
aircraft  dictate  that  the  aircraft  state  must  remain  within  specified  limits,  called 
the  aerodynamic  flight  enyelope,  given  by  F  6  [Vmin,  Vax],  7  e  [7min,7maxl, 
^  ^  [^min  5  ^max]  j  ^  G  [^min »  ^max] )  ^  ^  [<5min  y  ^max]  i  5  €  ,  ^max]  •  The  Set  G 

includes  all  states  which  are  not  inside  these  bounds. 

3  Safety  Analysis 

We  obtain  the  maximal  controlled  invariant  set  by  first  specifying  a  cost  function 
Jsa.ie{x^t)  whose  initial  condition  Jsafe(3:,  0)  =  l[x)  encodes  the  boundary  of 
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the  allowable  states  dG.  (The  function  l{x)  is  negative  inside  G,  zero  on  dG, 
and  positive  outside  G).  We  pose  the  problem  as  an  optimal  control  problem 
and  solve  two  coupled  Hamilton- Jacobi  equations  as  in  [3,17],  whose  solution 
describes  the  boundary  of  the  maximal  controlled  invariant  set  W*  and  the  safe 
set  of  control  inputs  Usafe{x)^  For  the  collision  avoidance  scenario,  we  can  find 
a  closed-form  solution  for  the  representation  of  W*.  In  the  flight  management 
system,  due  to  the  system’s  high  dimension,  we  analyze  the  safety  of  the  system 
by  projecting  the  system  onto  two-dimensional  subspaces  and  then  analyzing 
the  safety  of  the  system  within  each  projection.  The  safety  of  the  entire  system 
is  guaranteed  by  specifying  that  the  aircraft  remain  within  the  intersection  of 
these  two-dimensional  safe  regions,  which  is  a  subset  of  the  maximal  controlled 
invariant  set. 

3.1  Collision  Avoidance 

Due  to  finite  control  input,  there  are  certain  initial  configurations  of  the  two 
aircraft  for  which,  despite  their  best  efforts,  the  aircraft  will  eventually  violate 
G.  To  find  this  region,  we  use  the  method  of  [14],  but  obtain  an  analytical  solution 
due  to  the  cooperation  between  aircraft.  An  analytic  solution  arises  because  the 
computation  of  the  safe  control  law  results  in  Wsafe(^r)  =  [^safei,^safe2l  where 
^^safei  —  'Itsafe2j  meaning  that,  along  optimal  trajectories  Or  =  ^rsafe  is  constant 
and  thus  (3)  becomes  an  affine  system: 


0 

"y  ^safe  1  ff 

U  “1“  U  cos  ^rsafe 

^  tan  Wsafe  1 

0 

0 

^r  + 

Usin  0rsB.fe 

0 

0 

0_ 

0 

The  boundary  of  the  usable  part  on  dG  is  given  by  BUP  =  {(xp,  yp),  {—Xp,  —yp)), 
where  Xp  =  7^  0  and  yp  =  VI  -  cos Or.  Integrating  (6)  directly 

from  the  BUP,  and  eliminating  time: 

2 

{Xr{t)  +  Csin^r)^  +  (Vrit)  +  c(l  -  COS^r))^  =  ^5  -f  Cy/2{1  -  COS^^)) 

;  _ (2  (7) 

{Xr{t)  —  csin^r)^  +  (yr(0  ~  ^(1  —  COS^^))  —  ^5  +  C^/2{1  —  COS^r)j 
where  c  =  — — -  The  maximal  controlled  invariant  set  W*  =  {(-^i  U  U 

ptan^>niax 

A3)  n  X2}  is  a  function  of  x^,  where  the  sets  Xj  =  {x^  j  hj{x^)  <  0},  and 

hi(x^)  =  (Xr  +  csin^r)^  +  {yr  +  c(l  -  cos^^))^  -  ^5  +  c-/2(l  -  COS^r)) 

h2{x^)  =  {Xr  -  csin0^)^  +  {yr  -  c(l  -  cosOr))^  -  (5  +  cy2(l~^^^cos^^ 
hsix^)  =  -Xr{l  -  cos  Or)  +  2/r  sin 

h4{xj)  =  x'^  -\-yl  —  2b 

hsix^)  =  Xr  sin  Or  +yr{^  -  cos  Or)  • 

These  sets  are  depicted  in  Figure  1  (projected  onto  {xr,yr)  for  a  given  Or)  and 
Figure  2.  The  control  law  on  the  boundary  of  W*  is  given  by 
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Fig.  1.  Unsafe  region  for  a  given  Or  Fig.  2.  Maximal  Controlled  Invariant  Set  W* 


0max 


'^safe(^r)  —  \ 


0max  J 

^max 

^max 


fer  I  (^ifer)  =  0  A  h^ix^)  <  0  A  /isfer)  <  0)} 

.  (9) 

fer  I  (^2fer)  =  0  A  /l3(^r)  <  9  A  h^{x^)  >  0)} 


Details  of  this  analysis  are  presented  in  [18]. 


3.2  VSTOL  FMS  Model 

The  safety  analysis  for  the  VSTOL  FMS  in  TRANSITION  mode  follows  [2].  Due 
to  the  high  dimensionality  of  the  system,  we  analyze  the  safety  of  the  system 
in  two-dimensional  projections  onto  the  (V,7),  {9,9),  and  {5,6)  spaces.  We  then 
intersect  these  results  to  form  the  controlled  invariant  set  W*  =  O 

^{6, 9)  ^^{5,5)}^  where 


I  (^^in  ^  F  <  ^^ax  )  A  (7min  ^  7  ^  7max)} 

^^(9,9)  I  (^min  9  ^  ^max)  A  (^min  ^  9  ^max)  A 

{^~y/‘^{9  —  ^min)^2max/  J  9  <  yj‘2.{9  —  ^max)'^^2min/'^^  |  (10) 

^(5,5)  “  1^—  I  (^min  ^  ^  <^max)  A  ^  ^  ^  ^max)  A 

(“■\/^(^  ■“  ‘^min)'^3max  S  ^  \/2(<^  —  ^inax)^3min) ^  • 


The  set  valued  control  law  t4afe(^)  restricts  the  control  along  certain  boundaries 


of  W\  With  = 


9t 


(5.= 


2li3, 


+  (^r 


2'U'2  min 

^safe 


+  9, 


9o  = 


9t 


2'li2max 

is  defined  as: 


+  9, 


= 


2^3 1 


4-  6j 


and 
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< 


=  ^(q;,  5,  V^axj 't)  s.t. 

Ui  =  T'(q:,  (5,  V, '7niin) 


(T  cos(q;  +  5)  >  aDVmin^(l  +  b{l  +  ca)‘^) 

+mg  sin  7) 

(T cos(a  +  S)  <  +  ca)‘^) 

-\-mg  sin  7) 

{Tsm{9  -  7nain  +  ^)  >  -aL^^(l  +  c{9  -  7min)) 


Ul  —  7max) 


W2  =  'W2min 
U2  =  W2max 
W2  <  0 
W2  >  0 


+mg  COS7min) 

S.t.  (T sin(0  -  7max  +  <5)  <  -ax,V"2(l  +  c{9  -  7max)) 

+mp  cos^Wx) _ 

when  {9  =  -y^2{9  -  6>niin)^2max/^)  A  (0  >  ^1) 
when  (0  =  y'2(^-0max)ti2min/‘^)  A  (^  <  6'2) 
when  =  ^max)  A  (6/  <  ^1) 
when  {9  =  ^min)  A  (^  >  6>2) 


^3  =  1^3  min 
^3  =  "l^Smax 

W3  <  0 

^us>0 


when  (j  =  -y/2{s  -  <^min)^^3  max  )  A  (<5  >  5i) 
when  (5  =  \/2{5  —  <5max  )^3min)  A  ((^  <  S2) 
when  {5  =  ^^ax)  A  ((5  <  <5i) 
when  (5  =  <5min)  A  (5  >  S2)  . 


(11) 


4  Nonlinear  Constrained  Optimization 

We  now  seek  to  solve  the  nonlinear  constrained  optimization  problem  (2).  In  the 
case  of  the  flight  management  system,  this  is  fairly  straightforward,  as  the  con¬ 
trolled  invariant  set  is  already  written  as  an  intersection  of  inequality  constraints. 
The  collision  avoidance  scenario,  however,  results  in  an  expression  for  the  maxi¬ 
mal  controlled  invariant  set  which  is  represented  as  a  combination  (not  just  the 
intersection)  of  many  inequalities.  In  order  to  use  the  optimization  framework 
above,  we  introduce  a  mixed-integer  programming  framework  to  represent  the 
maximal  controlled  invariant  set  as  an  intersection  of  inequality  constraints. 

4.1  Collision  Avoidance 

Binary  variables  61,62^  S3  are  introduced  for  each  of  the  regions  Xi ,  X2  and  X3 
(see  Figure  1)  [6].  By  adding  constraints  which  involve  the  binary  variables,  we 
can  reformulate  the  inequalities  which  express  W*  as  cw(^r)  ^  ^  from  (2). 

<^i(^r)  =  1  ^  ^i(^r)  ^  0,  miSi(x^)  <  hi{x^)  <  Mi(l  -  5l(^^)) 

S2{xJ)  =  l  ^  h2{x^)  <  0,  m2S2{x^)  <  h2(Xr)  ^  ^2(1  -  <^2(^r))  (^2) 

S2(Xr)  =  1  ^  hslx^)  <  0,  msSslx^)  <  <  ^3(1  -  (^3(^r)) 

5i(^.)+^2(^.)  +  ^3(^.)<2  (13) 

Inequalities  (12)  express  the  sets  Xi,X2,X3  with  mj  =  minx^Xj  and  Mj  = 
rnaxi  Aj.  Figure  3  shows  the  possible  ((^1,(^2, ^3)  for  a  given  9r.  Thus  (13)  in 
conjunction  with  the  constraint  that  the  system  remain  outside  of  G  (x^  +  2/^  ^ 
25)  can  be  used  to  represent  W* .  The  continuous  system  is  now  a  differential 
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Fig.  3.  Possible  ((5i,(52,<53)  combi-  Fig.  4.  Optimization  without  safety  con- 

nations  for  a  given  Or  straints  over  a  shortened  time  horizon 

algebraic  system,  with  nonlinear  dynamics  subject  to  algebraic  inequality  con¬ 
straints  involving  the  states  and  control. 

The  continuous  dynamics  in  inertial  coordinates  are  discretized  with  the  ex¬ 
plicit  Euler  formula  over  N  time-steps  of  size  We  wish  to  minimize  the  cost 
function 

N 

•^perf  =  +  ('02, A:  —  02, o)^  +  (14) 

A:=l 

which  penalizes  deviations  from  the  aircrafts’  original  headings  while  minimizing 
control  effort.  This  minimization  is  subject  to  the  following  constraints,  which 
involve  the  inertial  and  relative  equations  of  motion,  safety  constraints,  and  final 
state  constraints  which  return  both  aircraft  to  their  original  headings. 

^i,fc  =  +  VAsin0i,fc  Xr^k  =  sin0i,fc(a:2,fc  -  xi^k) 

yi,k  =  2/i,fc-i  +  V/^cos'ipi^k  +cos0i,fc(?/2,fc  -  yi,k) 

0i,fc  ^  0i,fc-i  +  fzictanui,fc  =  -  cos0i,fc(x2,A:  .  . 

X2,k  —  X2^k-l  +V/^smip2,k  +sin0i,fc(y2,A:  -  yi,fc)  ^  ^ 

2/2,fc  =  y2,k-l  +  VZtcOS'lp2,k  Or,k  =  01,fc  -  02,fc _ 

02, fc  =  02,A:-1  +  ^^tanU2,k  Vfc  ~  5  +  01/2(1  —  cos  Or) 

hi,k  =  {Xr,k  +csinOr^k)'^  miSi^k  <  hi^k  <  M/l  -  5i,fc) 

+(2/r,fc  +  c(l  -  cos  9r,k))‘^  -  r| 

^2,fc  =  {Xr,k  —  C  sin  Or  ^k)"^  ^2^2, k  ^  ^2, A:  ^  -A^2(l  “  <^2,A:)  /-.^n 

Hyr,k-C{l  ~  COS  Or, k))^  -rl 

h3,k  =  -Xr,k{^  -  +  yr,k  sin  Or, k  TTlzS^^k  <  h^^k  <  ^3(1  -  S^^k) 

2  >  (^l,A:  +  ^2,A;  +  ^3,k 


25  <  x^  j^  -1-  yl  j_  01^^  =  010  02, N  =  02,0 


(17) 
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4.2  VSTOL  FMS 


Unlike  the  collision  avoidance  scenario,  the  restrictions  for  safety  in  the  VSTOL 
FMS  are  already  represented  as  the  intersection  of  inequalities.  We  discretize  the 
continuous  system  (5)  through  an  explicit  Euler  formulation  over  N  time  steps 
of  length  /a.  We  wish  to  minimize  the  cost  function 


•^perf  — 


(18) 


which  penalizes  large  control  inputs.  This  minimization  is  subject  to  the  equa¬ 
tions  of  motion,  initial,  and  final  constraints.  The  initial  state  constraints  fix 
inertial  positions  and  velocities  at  xq,  xq,  zq,  zq.  Final  constraints  force  the  air¬ 
craft  to  reach  a  minimum  desired  velocity  Vf  and  desired  altitude  z/  by  the  final 
time  tiv.  Additionally,  final  state  constraints  on  5  and  5  maintain  continuity  of 
the  hybrid  system  across  the  switch  from  TRANSITION  mode  to  CTOL  (Con¬ 
ventional  Take-Off  and  Landing)  mode.  (The  system’s  continued  trajectory  in 
CTOL  mode  is  not  presented  here). 


Xk  =  Xfc-i  +  ^Xk 
Zk  =  ^Zk 

Ok  =  0k~\  +  ^Ok 
Ok  =  0k~\  +  ^U2,klJ 
dk  —  5k-\  + 

8k  =  8k-i  +  ^u^,k 


Vk  =  y/il  + 

7fc=tan  ^(zk/xk) 
ak  =  Ok  —  7fc 

Dk=aDV^{l  +  h{l  +  cak)^) 
Lk  =  aLVfc  (1  +  cafc) 


VN>Vf 

ZN  >  Zf 

Sn  ~0 

5^=0 


(19) 


Xk 

h 


Xk-l 

4-1 


+ 


RiOk. 


At 

M 

—Dk-i 

Lk-l 

+ 

i) 

Ui^k-1  cos  4-1 
^i,fc-isin4-i  —  ^^2,^-1^ 

- 

0 

Mg 

(20) 


-y/2{0k  -  4min)^2,fcn,ax/^  <  4  <  \/2(4  "  4max)^2,fcnim/^  (21) 

“  ^  4  ^  \/2(4  “  4max)'^3,fcjnin 


5  Results 

We  modeled  both  examples  in  GAMS,  a  programming  environment  which  in¬ 
vokes  prescribed  solvers  for  mixed-integer  nonlinear  programs  (MINLPs)  and 
nonlinear  programs  (NLPs)  [19].  The  MINLP  solver,  DICOPT  [20],  successively 
solves  NLPs  and  mixed-integer  linear  programs  (MIPs)  until  the  solution  con¬ 
verges  to  its  optimum  value.  DICOPT  used  two  nonlinear  solvers,  CONOPT  [21] 
and  rSQP  [22],  as  well  as  the  MIP  solver  CPLEX  [23].  The  plain  NLP  problems 
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used  the  nonlinear  solver  CONOPT.  The  three  collision  avoidance  scenarios  all 
begin  at  the  same  initial  state  Xq  —  [007r/2  877r]^,  but  utilize  different  con¬ 
trollers  to  address  (1)  optimization  with  safety  restrictions,  (2)  tracking  with 
safety  restrictions  (no  optimization),  and  (3)  optimization  without  safety  re¬ 
strictions.  The  two  optimizations  were  computed  in  approximately  140  seconds 
and  4809  iterations,  and  1  second  and  3  iterations,  respectively,  on  a  Dell  400 
MHz  single  processor  with  128MB  RAM.  The  two  VSTOL  scenarios  compare 
optimization  with  safety  restrictions  and  tracking  with  safety  restrictions.  The 
nonlinear  optimization  completed  in  330  seconds  and  2356  iterations  on  a  SunUl- 
tra60Creator3D  with  384MB  RAM. 

5.1  Collision  Avoidance 

Optimization  with  Safety  Constraints.  The  entire  system  (14)-(17)  was 
optimized,  constraining  the  optimal  solution  to  lie  within  the  range  of  allowable 
controls  C4afe  well  as  within  W*  at  each  time  point.  The  optimal  trajectory 
smoothly  navigates  both  aircraft  in  W*,  and  the  resultant  control  does  not 
chatter  (Figure  5).  This  method  produces  a  well-behaved  control  law  and  smooth 
trajectories  for  the  aircraft. 


time  in  seconds 

Fig.  5.  Optimization  with  safety  constraints:  Trajectory  in  relative  coordinates  and 
control  history 


Tracking  with  Safety  Override,  We  contrast  the  results  from  the  above 
method  with  a  simple  method  used  in  [2]  for  longitudinal  envelope  protection.  In 
this  method,  a  tracking  control  law  is  overridden  when  necessary  with  the  control 
law  to  enforce  safety.  The  continuous  system  is  subject  to  actuator  saturation, 
and  the  state  of  the  system  is  continually  examined,  enabling  the  safety  controller 
(9)  to  override  the  tracking  controller  when  the  system  encounters  the  boundary 
of  W\ 
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^perf(^)  — 


Y  Ttan  ~ipi{0))) 

9  [tan-i(-A('^2  -  V^2(0))) 

t^safe  (^) 


xeW* 

otherwise 


(22) 


The  nonlinear  inversion  tracking  control  law  places  the  poles  of  the  error  dy¬ 
namics  on  the  negative  real  axis  at  —A  =  —1.5.  While  this  approach  is  appealing 
in  its  simplicity,  in  practice  it  is  problematic  due  to  the  chattering  in  the  control 
law  when  the  system  switches  from  the  tracking  control  law  to  the  safety  control 
law  (Figure  6).  The  chattering  results  from  the  fact  that  the  control  law  chosen 
for  tracking  is  often  completely  contradictory  to  the  control  law  necessary  for 
safety. 


Fig.  6.  Tracking  with  safety  override:  Trajectory  in  relative  frame  and  control  history 


Optimization  Without  Safety  Constraints.  For  completeness,  the  system 
is  also  compared  to  the  one-step  nonlinear  optimization  method  used  in  [12].  The 
system  (14,15,17)  is  optimized,  maintaining  aircraft  separation  and  constrain¬ 
ing  u  e  U.  This  requires  only  an  NLP  (not  MINLP)  solver  since  the  maximal 
controlled  invariant  set  is  ignored. 

For  generic  initial  conditions  and  time  horizons,  there  is  no  guarantee  of 
safety,  of  remaining  within  W*.  As  shown  in  Figure  4,  the  optimal  control  law 
leads  the  aircraft  right  to  the  boundary  of  G.  While  maintaining  aircraft  sepa¬ 
ration  for  the  time  over  which  the  system  is  optimized,  the  aircraft  are  left  in  an 
orientation  which  will  inevitably  result  in  a  violation  of  the  minimum  aircraft 
separation  (4),  demonstrating  the  advantage  of  two-step  controller  synthesis. 
Separating  the  safety  and  performance  goals  into  a  two-stage  optimization  prob¬ 
lem  enforces  safety  over  any  time  horizon. 

The  computational  difficulties  associated  with  MINLPs  make  the  NLP  sce¬ 
nario,  with  minimum  separation  inequality  (4)  but  no  safety  restrictions  W*, 
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appealing  for  cases  in  which  we  know  ahead  of  time  that  the  time  horizon  we 
optimize  over  is  “long  enough”  to  complete  the  conflict  avoidance  maneuver.  For 
a  time  horizon  of  100  seconds  CONOPT  solved  the  system  in  23  seconds  and 
676  iterations,  considerably  less  than  the  MINLP  solver  used  in  the  optimization 
with  safety  constraints. 

5.2  VSTOL  FMS 

We  perform  a  similar  comparison  of  two  multiobjective  methods  for  the  flight 
management  system.  The  aircraft  begins  at  [rro  zq  zqY  =  [A  40  18  0]^  in 
both  cases.  We  compare  the  two-step  controller  synthesis,  optimizing  perfor¬ 
mance  within  the  safety  restrictions,  with  the  method  used  in  [2],  overriding  a 
tracking  control  law  with  the  safety  control  law  when  necessary. 

The  system  (18)- (21)  is  optimized  and  plotted  (solid)  against  the  trajectory 
obtained  from  tracking  (dashed)  in  Figures  7  through  10.  The  optimized  trajec¬ 
tory  is  smoother  than  the  tracking  trajectory,  does  not  cause  pitch  oscillations 
(as  the  tracking  trajectory  tends  to  do),  and  does  not  chatter  despite  reaching 
saturation  in  the  thrust  input.  The  considerable  difference  in  nozzle  angle  tra¬ 
jectories  (Figure  8)  could  result  from  the  fact  that  our  model  does  not  account 
for  interactions  with  the  ground. 


Fig.  7.  Trajectories  in  longitudinal  plane  Fig.  8.  Nozzle  angle  and  thrust  time  his¬ 
tories 


6  Conclusion 

The  results  of  this  paper  serve  to  motivate  the  problem  of  developing  computa¬ 
tionally  efficient  methods  for  multiobjective  controller  synthesis  in  hybrid  sys¬ 
tems.  We  have  shown  that  for  nonlinear  continuous  state  systems,  it  is  feasible  to 
combine  safe  set  computation  with  constrained  nonlinear  programming  in  order 
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Fig.  9.  Trajectories  in  (V,7)  Projection  Fig.  10.  Trajectories  in  {9,0)  Projection 


to  compute  solutions  which  satisfy  both  safety  and  performance  goals.  However, 
there  are  a  number  of  issues  which  need  to  be  addressed.  The  current  solvers  are 
very  sensitive  to  the  initial  values  of  the  state  and  control  trajectories,  so  if  these 
solvers  were  to  be  used  in  practice  today,  good  intuition  is  needed  to  provide  an 
initial  iterate.  Our  results  from  the  collision  avoidance  scenario  could  be  readily 
extended  to  a  higher  number  of  aircraft  by  examining  the  relative  separation 
between  each  aircraft  pair.  However,  for  this  to  be  a  feasible  method  to  obtain 
optimal  trajectories,  solving  a  mixed-integer  nonlinear  program  for  nonlinear, 
trigonometric  functions  needs  to  become  a  simpler  process.  The  representation 
of  the  maximal  controlled  invariant  set  in  closed  form  is  also  required:  currently, 
we  can  do  this  only  for  systems  for  which  we  can  solve  Hamilton’s  equations 
analytically.  For  more  complicated  systems,  an  under  approximation  of  the  safe 
set  with  a  simpler  representation  is  required.  Finally,  we  are  now  extending  these 
techniques  to  systems  with  multiple  discrete  modes  (the  full  hybrid  model  of  the 
VSTOL  aircraft),  which  requires  optimization  across  the  mode  switch  as  well  as 
within  each  mode. 
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Abstract.  The  paper  concerns  the  representation  of  continuous- variable 
discrete- time  systems  with  quantised  input  and  state.  It  shows  that  the 
autonomous  quantised  system  is  represented  by  the  Probenius-Perron 
operator  and  the  non- autonomous  by  the  Foias  operator.  A  finite  and 
complete  approximation  of  the  Probenius-Perron  operator  is  given  by 
an  automaton  which  turns  out  to  be  identical  to  the  discrete  abstrac¬ 
tion  of  the  quantised  system  that  is  currently  studied  in  the  literature 
on  verification  or  diagnosis  of  hybrid  systems.  Hence,  the  paper  shows 
a  connection  between  the  mathematical  literature  and  hybrid  systems 
research.  As  a  result  of  this  connection  it  is  shown  that  the  abstraction 
converges  to  the  continuous  system  for  finer  quantisation.  The  paper 
ends  with  presenting  a  method  for  the  computation  of  abstractions  that 
guarantees  the  completeness  of  the  resulting  model. 


1  Introduction 

This  paper  concerns  quantised  systems  (Figure  1),  which  are  a  specific  type  of 
hybrid  systems.  The  injector  and  the  quantiser  are  interfaces  between  the  nu¬ 
merical  signals  of  the  continuous- variable  system  and  the  symbolical  values  that 
serve  as  input  or  output  of  the  quantised  system.  The  motivation  for  consid¬ 
ering  such  systems  comes  from  process  supervision,  where  the  controller  of  a 
continuous  system  has  only  access  to  discrete  input  and  outputs. 


Fig.  1.  Quantised  system. 


Quantised  systems  have  been  studied  recently  in  the  literature  on  verifica¬ 
tion  of  discrete  control  algorithms  or  on  process  diagnosis  where  the  quantised 
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system  is  replaces  by  a  discrete-event  model,  which  refers  only  to  the  symbolic 
signals  [w]  and  [x]  but  does  no  longer  include  continuous- variable  elements.  This 
model  is  called  a  qualitative  model  of  the  continuous- variable  system  or  a  discrete 
abstraction  of  the  quantised  system. 

The  main  aim  of  this  paper  is  to  show  that  quantised  systems  can  also  be 
dealt  with  as  a  nonlinear  system  and  studied  by  means  of  methods  that  have 
been  developed  in  mathematical  systems  theory.  The  main  idea  is  to  consider 
the  set  of  all  states  x  of  the  continuous-variable  system  that  have  the  same 
quantised  value  [x]  and  to  follow  the  ensemble  of  all  trajectories  that  start  from 
this  set.  Such  trajectory  ensembles  can  be  described  by  the  Probenius-Perron 
operator  (FPO),  which  has  been  introduced  in  the  analysis  of  chaotic  systems. 
This  paper  shows  that  autonomous  quantised  systems  can  be  represented  by  the 
FPO.  Hence,  results  from  mathematical  systems  theory  can  be  directly  applied 
to  quantised  systems.  First,  it  is  shown  that  discrete-event  models  of  quantised 
systems  that  are  used  in  the  hybrid  system  literature  are  discrete  approximations 
of  the  FPO.  Hence,  well-known  properties  of  discrete  approximations  of  the  FPO 
can  be  used  to  prove  the  convergence  of  the  discrete  abstraction  for  increasing 
resolution  of  the  quantiser.  Second,  a  method  for  computing  complete  qualitative 
models  is  derived  by  using  the  idea  of  hyperbox  cell-to-cell  mapping. 

Relevant  literature.  There  are  two  lines  of  research  relevant  for  this  study. 
The  first  concerns  the  modelling  of  hybrid  systems  and  their  application  to  con¬ 
trol  tasks.  In  order  to  overcome  the  difficulties  brought  about  by  the  complexity 
of  hybrid  systems  automat  a- theoretic  descriptions  have  been  proposed  in  [9] 
and  [11]  for  quantised  discrete-time  continuous  systems  and  [2],  [5]  or  [10]  for 
discrete-event  quantised  systems.  The  other  line  of  research  concerns  the  math¬ 
ematical  study  of  nonlinear  and  chaotic  systems.  The  FPO  has  been  studied  to 
analyse  the  evolution  of  densities  of  nonlinear  transformations  throughout  the 
last  decades.  About  40  years  ago  a  finite  approximation  method  for  the  FPO 
has  been  suggested  in  [13].  Several  years  later  it  was  shown  in  [7]  for  scalar 
systems  and  in  [1]  for  multi-dimensional  systems  that  the  approximate  operator 
converges  to  the  FPO. 

This  paper  combines  both  lines  of  research  and  applies  results  on  the  FPO 
to  the  quantised  system. 

Structure  of  the  paper.  The  main  idea  of  the  presented  approaches  is  to 
consider  probability  density  functions  in  the  state-space  rather  than  single  states. 
In  Section  2  the  temporal  evolution  of  such  density  functions  is  considered.  It 
is  shown  that  this  evolution  is  precisely  described  by  the  FPO.  Section  3  deals 
with  the  approximation  of  the  behaviour  of  the  quantised  system.  The  resulting 
qualitative  model  is  a  stochastic  automaton  given  in  Section  4.  It  is  shown  that 
this  automaton  is  the  result  of  a  discretisation  of  the  FPO.  Section  5  deals 
with  computational  aspects  of  this  discretisation.  A  fundamental  requirement 
in  process  supervision  is  to  obtain  a  complete  model,  which  requires  a  sound 
approximation  of  the  FPO.  A  method  is  presented  that  guarantees  soundness 
for  Lipschitz-constrained  systems. 
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2  Evolution  of  Probability  Densities  in  the  State-Space 

2.1  Results  from  Measure  Theory 

For  a  description  of  the  behaviour  of  the  quantised  system  essential  concepts 
from  measure  theory  are  needed.  For  a  more  detailed  introduction  the  reader  is 
referred  to  the  textbook  [6]. 

Consider  a  set  i?,  which  is  usually  the  IR’^,  and  a  family  a{Q)  of  subsets  of 
17.  This  family  is  called  a  cr-algebra,  if 

1.  17  G  cr(l7),  and  A  G  a{0)  =>  17\A  G  cr(17), 

2.  for  every  sequence  {A^},  G  C7(l7)  Ufc  ^  cr(l7) 

hold.  A  measure  is  a  function  ^  :  17  ^  IR"^  that  satisfies  )Li(0)  =  0  and: 

|t((J  Ak)  =  ^  i^{Ak)  if  Ai  nAj  =0,  i^j. 

k  k 

The  triple  (17,  (7(17), /Li)  is  called  a  measure  space  and  all  A  G  (7(17)  measurable 
sets.  A  commonly  used  measure  space  is  the  Borel  measure  space  (IR,  /u), 
where  the  Borel  cr-algebra  is  by  definition  the  smallest  (7-algebra  containing  all 
intervals  [a,  6]  on  IR  and  the  Borel  measure  is  given  by  fi{[a,h])  ^  h  —  a.  Its 
extension  to  higher  dimension  yields  the  space  (IR”,  /i^),  which  contains  all 

hypercubes  with  their  hypervolume  as  measure.  For  17  C  IR”  the  corresponding 
Borel  (7- algebra  is  denoted  as  B(17). 

Given  a  measure  space  (17,  (7(17),  //),  a  function  p:  ->  IR  satisfying  G 

(7(17)  or  equivalently  {A  :  p{A)  G  G  (7(17)  for  every  interval  C  IR  is  called 
measurable.  The  Lebesgue  integral  is  defined  for  every  measurable  function  and 
is  denoted  by  For  a  set  A  G  (7(17)  the  Lebesgue  integral  is  defined 

as  J^p{(jij)p,(doj)  =  l^(u;)p(cj)Ai(da;)  with  the  indicator  function  1^(2^)  that 
is  1  for  a;  G  A  and  0  otherwise.  The  Borel  measure  of  every  Borel  measurable 
set  A  can  be  expressed  as  the  Lebesgue  integral:  /i(A)  =  /i(c?cj). 

In  a  measure  space  (17,(7(17),^)  the  family  of  all  measurable  functions  p  : 
f?  ^  IR  for  which  ||p||i  =  \p{u;)\p.{duj)  <  oo  holds,  is  called  (17, (7(17), /i) 
(abbreviated  as  space).  A  sub-space  of  is  the  space 

D{n,a{n),fi)  =  {peL^  :  p  >  0,  ||p||i  =  1} 

of  all  density  functions,  i.e.  those  functions  that  satisfy  the  properties  of 
density  functions  in  probability  theory. 

2.2  Problem  Statement 

For  simplicity  of  presentation,  the  theory  is  developed  now  for  autonomous  quan¬ 
tised  systems  and  extended  to  non- autonomous  systems  in  Section  2.5.  The 
continuous-variable  system  is  described  for  a  given  measure  space  (17, (7(17), /u) 
by 


x{k-^l)  =  f{x{k)) 


(1) 
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with  X  E  n  and  f  :  Q  f2.  The  initial  state  ic(0)  is  unknown.  Instead  only  an 
initial  density  function  po{x)  e  D(n,a{Q),fi)  is  given. 

The  aim  is  to  find  the  probability  with  which  the  symbols  [a; (A:)]  describing 
the  quantised  state  appear  at  the  output  of  the  quantised  system  in  two  steps: 

1.  The  evolution  of  the  initial  density  function  is  described  as  sequence  p{x,  k) 
of  density  functions  over  the  discrete  time  k,  with  p{x,0)  =  po{x)  and 
p{x,  k)  G  D,  Vk.  This  problem  will  be  solved  in  in  Section  2.3. 

2.  Prom  the  sequence  of  density  functions  p{x,  k)  the  discrete  conditional  prob¬ 
ability  distribution  Prob([a;(A:)]  \po(x))  of  the  state  symbols  for  given  quan¬ 
tiser  is  derived.  This  problem  will  be  investigated  in  Section  2.4. 


2.3  The  Probenius-Perron  Operator 

Definition  1,  Given  a  measure  space  (i7,  cr(J7),/i)  and  a  non-singular  measure- 
able  transformation  f  :  Q  Q,  for  which  for  every  A  G  a{n)  with  p{A)  =  0  the 
relation  p{f  ^(>1))  =  0  holds.  Then  the  Frobenius-Perron  operator  P  : 
associated  with  f  is  defined  by: 

/  Pp((^)p{duj)  =  I  p{(jj)p{duj)  ,  for  all  A  G  a{Q)  .  (2) 

Ja 


Note  that  P  is  implicitly  defined  by  eqn.  (2)  as  an  operator  which  p  G  into 
Pp  G  L\ 

If  f  :  IR”  — >  in  is  a  diffeomorphism,  i.e.  if  f  is  bijective  and 

both  /  and  f~^  are  differentiable,  the  FPO  is  explicitly  given  by 


Pp{x)=p{f  i(a;))- 


(3) 


where  \{df~^ /dx)\  denotes  the  determinant  of  the  Jacobian  of  f~^  [6]. 

The  FPO  P  is  a  linear  operator.  One  of  its  most  important  properties  is 
that  it  is  a  density  operator,  i.e.  Pp{x)  G  D,  whenever  p{x)  G  D.  Hence,  the 
FPO  solves  the  first  problem  given  in  Section  2.2.  With  p(x,0)  =  po{x)  G  D 
eqn.  (3)  is  used  to  determine  the  evolution  of  this  density  function  recursively 
by:  p(x,  A:  +  l)  =  Pp{x,  k). 

Examples.  From  eqn.  (3)  the  FPO  of  /  :  IR'^  IR+,  f{x)  =  is  given 
by  Pp(x)  =  p{\/x)/{2y/x).  For  the  initial  density  po{x)  =  5  •  l[o.6,  o.s],  which 
describes  a  uniform  distribution  of  the  system  state  in  the  interval  [o!6,  0.8],  the 
FPO  yields 


p{x,  1)  =  Pp{x,0)  =  ■  1 

2y/X 

p{x,k)  =  P'‘p{x,0)  =  -^ 
(2y/x) 


[0.62, 0.82]  and 
J  ■  l[0.62fc,  0.82*=]  . 


As  another  example  the  FPO  of  a  linear  system  x(A:-l-l)  =  Ax  with  non-singular 
matrix  A  is  given  by  Pp{x)  =  p{A~'^x)  •  |  det 
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2.4  Representation  of  the  Autonomous  Quantised  Systems 

In  the  following  the  measure  space  (i?,  H(l7), /i^),  Q  C  IR"^  is  considered.  The 
quantiser  introduces  a  partition  of  Q  into  N  regions  QxO-) i  Qx(N)  such 
that  >  0,  i  =  1, N  holds.  According  to  this  partition  the  quantiser 

assigns  to  each  value  x{k)  a  discrete  value  [x{k)]  e  A4  with  Mx  =  {1,2, N} 
such  that  x{k)  G  Qx{i)  ^  [x{k)]  =  i  holds. 

In  terms  of  a  function  p(£c)  the  quantiser  defines  a  projection  to  the  subset 


N  -y 

An  =  {p(a:)  :  p(®)  =  ’  b"{x),  a;  e  IR}  C  ,  6*(x)  = 

t=l 

of  all  functions  that  can  be  written  as  finite  sum  of  some  functions  6*(x). 
Definition  2.  A  projector  is  an  operator  Qn  :  An  with: 


N 


qnp  ~ 


i=l 


-I 

jQAi) 


p{u))ix^{dio) 


(5) 


According  to  this  definition,  the  projection  is  such  that  the  weight  Xi/ p^{Qx{i)) 
of  each  simple  function  iQ^(t)  is  the  mean  value  of  p{x)  in  the  region  Qx(0- 
A  discretiser  is  associated  with  the  projector  which  maps  a  density  function 
p{x)  to  an  A-dimensional  discrete  probability  distribution  Prob([a;(A:)])  G 
with  =  {pD  e  [0,  1]^  :  =  1}^  ^here  is  the  i-th  element  of  the 

N-vector  po- 

Definition  3.  The  operator  Dn  ’  T)  — >  W^,  DnV  —  (-^i,  •  •  • ,  XnY  with  Aj  given 
by  eqn.  (5)  is  called  (density)  discretiser. 

Hence,  the  autonomous  quantised  system  is  described  by 

p(a;,fc+l)  =  Pp(®,  A:),  p{x,Q)=Pq{x)  (6) 

Prob([x(A:)]  |  po{x))  =  Dnp{x,  k)  ,  (7) 

with  initial  density  function  po{x).  That  is,  it  is  represented  by  the  FPO  P  of 
the  continuous-variable  system  and  by  the  discretiser  Dn  associated  with  the 
quantiser. 

Example.  Consider  again  f{x)  =  in  p)  with  po{x)  =  5  •  l[o.6,  o.s] 

and  the  partition  Qx(l)  =  [0,  0-^),  &(2)  =  [0-5,  1),  Qx(3)  =  [1,  oo).  Then 
eqns.  (6)  and  (7)  yields  (Figure  2): 

DnP{x,0)  =  (0  1  0)'  D^pix^O)  =  (0.4645  0.5355  0)'  etc. 
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Fig.  2.  Sequence  of  density  functions.  Fig.  3.  Explanation  of  Ulam’s  method. 


2.5  Extension  to  Non- autonomous  Systems 

The  aim  of  this  section  is  to  define  an  operator  similar  to  the  FPO  for  non- 
autonomous  systems.  As  this  operator  is  formulated  as  an  operator  acting  on 
measures,  first  the  FPO  on  measures  is  given. 

In  Section  2.1  the  Borel  measure  has  been  written  as  fi{A)  =  An 

important  result  of  measure  theory,  the  Radon-Nikodym  theorem,  says  that  for 
any  measure  in  (12,  /i)  satisfying  fi{A)  =  0  =>  i/(A)  =  0  there  exists  a 

non-negative,  integrable  function  p:  Q  ^  \R  such  that 

v{A)  =  f  p(uf)fi{dLj)  .  (8) 

J  A 

Such  a  measure  u  is  said  to  be  absolute  continuous  to  p.  This  result  means  that 
in  a  certain  sense  a  density  corresponds  to  a  measure  and  vice-versa.  Not  every 
measure  can  be  represented  by  a  density  function,  but  every  density  leads  to  a 
measure  that  is  absolute  continuous  to  the  Borel  measure.  As  a  consequence,  the 
FPO  can  be  formulated  as  an  operator  transforming  one  measure  into  another 
instead  of  an  operator  transforming  one  density  into  another  as  done  in  the 
previous  sections. 

Consider  all  finite  measures  on  i.e.  all  measures  for  which  p{A)  < 

oo  holds  for  all  A  G  cr(i7),  and  denote  the  space  of  all  these  measures  by  A4. 
Then  the  FPO  P  :  A4  M.  on  measures  is  given  by 

Piy{A)  =  [  lA{f{io))iy{duj)  .  (9) 

Jn 

For  measures  in  the  form  (8)  and  non-singular  transformations  /  the  FPO  on 
measures  becomes  the  FPO  in  the  form  of  eqn.  (2)  [6]. 

Consider  now  the  non- autonomous  system 


x{k^l)  =  f{x{k),u{k)) 


(10) 
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with  X  e  O,  u  ^  ^  and  f  :  Q  Q  together  with  the  quantiser  and  injector 

as  shown  in  Figure  1.  In  the  following  it  is  assumed  that  1?  C  IR”^,  ^  C  IR’^  and 
both  Q  and  ^  are  closed  and  Borel  measurable.  Furthermore,  for  every  fixed 
u  G  ^  the  transformation  f{x,u)  is  assumed  to  be  continuous  in  x  and  for 
every  fixed  x  e  O  measurable  in  u.  Borel  measures  used  in  the  following  are 
denoted  by  for  B{Q)  and  for  B{^). 

The  injector  is  defined  similar  to  the  quantiser  by  a  partition  of  ^  into  M 
regions  Qu(l),  •  •  • ,  such  that  >  0,  I  ^  holds.  For 

given  discrete  input  I  e  Nu  with  A/'u  =  {1, M}  the  injector  chooses  a  value 
u{k)  from  Qu{l)  randomly  according  to  a  given  time-invariant  distribution.  That 
is,  for  every  I  ^  Mu  ^  probability  measure 

f  ioT  B  G  B{Qu{l))  (11) 

Jb 

is  given  which  is  the  same  for  all  k. 

Under  the  assumption  that  the  random  vectors  a? (0),  ti(0 ),«(!),.. .  are  inde¬ 
pendent  of  each  other,  for  each  I  e  Mu  the  following  operator  can  be  defined: 


Definition  4.  [6]  The  operator  :  M  M  associated  with  the  system  (10) 
for  given  measure  is  defined  by 


P‘n(A)  = 


(12) 


with  pG  M  and  A  G  B{Q).  This  operator  is  called  Foias  operator. 

Remark.  In  controlled  systems  the  independence  assumption  concerning 
the  initial  states  and  the  inputs  is  usually  not  satisfied.  It  has  not  yet  been 
investigated  whether  the  Foias  operator  can  be  extended  avoiding  the  use  of  a 
product  of  measures  as  in  eqn.  (12)  requiring  the  independence  assumption. 

By  means  of  the  Foias  operator  the  non-autonomous  quantised  system  with 
initial  state  measure  pc{A)  =  f^po(u^)p^(dcj)  is  described  by 

=  tik{A)  =  (13) 

Prob([a?(/!:)]  |po(ic),[w(0)],...,[w(/c)])  =DjvK®,A:)  ,  (14) 

for  any  A  G  B{Q)  assuming  that  the  transformation  f{x,u)  is  such  that  all 
measures  obtained  by  application  of  eqn.  (13)  are  absolute  continuous  to  the 
Borel  measure  According  to  eqns.  (13),  (14)  the  non-autonomous  quantised 
system  is  represented  by  the  set  of  Foias  operators  obtained  for  [u]  G  Mu 
each  depending  on  the  corresponding  measure  introduced  by  the  injector 
and  the  discretiser  Dm  associated  with  the  quantiser. 

The  difficulty  is  that  the  representation  of  the  non-autonomous  system  (13)- 
(14)  is  described  by  a  transformation  of  measures  rather  than  by  a  transfor¬ 
mation  of  densities.  The  Foias  operator  can  be  transformed  into  an  operator 
on  densities  similar  to  the  FPO  in  eqn.  (3)  but,  in  contrast  to  the  FPO,  this 
representation  still  depends  upon  the  given  densities. 
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3  Approximate  Representation  of  Quantised  Systems 

3.1  Approximation  of  the  FPO 

In  the  previous  section  the  FPO  and  the  Foias  operator  as  representations  of  the 
quantised  system  were  introduced.  However,  a  closed  form  of  the  FPO  as  given 
in  the  examples  in  Sections  2.3  and  2.4  can  only  be  found  for  simple  systems, 
and  the  Foias  operator  cannot  even  for  simple  systems  be  given  explicitly. 

Hence,  if  the  FPO  should  be  used  to  solve  process  supervision  tasks,  an 
approximation  of  the  FPO  is  needed  that  can  be  found  explicitly  for  arbitrary 
transformation  /.  The  approximation  presented  in  this  section  is  based  on  the 
restriction  that  the  FPO  should  not  be  applicable  to  all  functions  but  only  to 
those  which  can  be  represented  by  a  finite  sum  of  indicator  functions  (so-called 
simple  functions).  More  precisely,  as  before,  the  measure  space  (J?,  5(i7), 

O  C  is  considered  with  a  partition  of  J?  into  N  regions  ( 1 ),..., 

and  the  subset  of  all  functions  as  in  eqn.  (4)  is  used.  In  order  to  apply  the 
FPO  to  the  quantised  system,  this  partition  is  set  to  the  partition  introduced 
by  the  quantiser  (cf.  Section  2,4). 

Definition  5.  The  quantised  Frobenius-Perron  operator  with  respect  to  f  is 
defined  as  the  operator  :  An  An  with: 

PNb’ix)  =  ^Prob(i|j)  •  b‘(a:)  ,  Prob(i|j)  =  ^  •  (^5) 

i=i  A*  (^xVJ)) 

The  quantised  FPO  has  been  introduced  in  [13]  and  is  also  called  Ulam’s  piece- 
wise  constant  approximation  of  the  FPO.  The  conditional  probabilities  Prob(i|  j) 
define  a  Markov  chain  with  the  state  set  {1, .  * .  A^}  or,  in  terms  of  qualitative 
modelling,  an  autonomous  stochastic  automaton  (cf.  [9],  Section  4.2). 

Figure  3  explains  the  meaning  of  eqn.  (15).  The  conditional  probability 
Prob(i|j)  describes  the  probability  that  the  successor  state  of  the  continuous- 
variable  system  (1)  is  in  Qx(i)  if  it  is  known  that  the  system  state  is  currently  in 
Qi(j)-  It  is  given  by  the  ratio  between  the  measures  of  the  set  f~^{Qx{i))(~^Qx{j) 
and  of  the  entire  region  Qx{j)- 

The  following  theorem  describes  the  relation  between  the  FPO  and  the  quan¬ 
tised  FPO  using  the  projector  Qn  oi  Definition  2. 

Theorem  1.  [1]  For  all  p{x)  G  An  the  relation  Pnp{x)  =  QnPp{x)  holds. 

According  to  this  theorem,  the  map  Pnp{x)  of  any  function  p{x)  6  An  deter¬ 
mined  with  the  quantised  FPO  Pn  is  the  same  as  the  projection  of  the  precise 
map  Pp{x)  by  the  continuous  FPO  to  Note  that  the  theorem  only  holds  in 
terms  of  the  projection  to  An-  Hence,  the  theorem  means  that  the  weights  of 
the  P{x)  determined  by  Pn  are  the  same  as  the  obtained  by  application  of 
eqn.  (5)  to  Pp{x). 

Theorem  2.  [7]  For  all  p{x)  e  An  the  relation  Pnp(x)  ^-^^Pp{x)  holds. 

This  important  result  means  that  for  increasingly  finer  partition  the  quantised 
FPO  converges  to  the  FPO,  for  any  transformation  /. 
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3.2  Extension  to  Non-autonomous  Systems 

As  before  the  measure  spaces  (i?,  H(i7),  /i”)  with  Q  C  and  with 

^  C  IR""  are  considered.  In  addition  to  the  partition  introduced  in  the  previous 
section,  further  partitions  of  each  could  be  introduced  to  approximate  the 
Foias  operators.  Instead,  as  the  input  set  is  already  partitioned,  each  input 
distribution  p\u)  as  introduced  in  eqn.  (11)  is  approximated  by  a  single  indicator 
function  using  the  projector  of  Definition  2: 


Qip\u) 


M’"(eu(0 


p^iQuii))  ■ 


After  partitioning  the  state  set  Q  as  before  the  following  operator  can  be  defined: 


Definition  6.  The  quantised  Foias  operator  with  respect  to  the  transformation 
f  is  defined  as  the  operator  :  An  An  with: 


N 

PnV{x)  =  '^Prob{i\j,l)  •  P(x),  Prob(i|j,/) 

i=l 


and  QxuUJ)  QxU)  x  Qu{l)- 


y^-^^{r\Qx{i))nQxu(Tl)) 

y^+^{Qxu{jJ)) 

(16) 


Figure  3  can  also  be  used  to  explain  the  quantised  Foias  operator.  f~^{Qx{i)) 
defines  a  set  of  states  and  inputs  in  the  compound  state  and  input  set  j?  x 
Eqn.  (16)  is  the  relation  of  the  measures  of  the  subset  of  f~^{Qx(i))  lying  in 
QxuUJ)  and  the  entire  region  QxuiJJ)- 

Theorems  1  and  2  seem  to  hold  for  the  quantised  Foias  operator,  although 
this  has  not  yet  been  proved  in  literature. 


4  Qualitative  Modelling 

4.1  Modelling  Aim 

The  methods  described  in  the  previous  section  were  developed  in  the  literature  on 
nonlinear,  especially  chaotic  systems.  Hence,  they  were  applied  with  partitions 
as  fine  as  necessary  for  obtaining  numerically  precise  solutions  for  stationary 
densities  etc.  (cf.  e.g.  [3],  [12]). 

In  contrast  to  this,  the  aim  of  this  paper  is  to  obtain  a  model  for  process 
supervision  purposes  while  leaving  the  partition  as  rough  as  possible  or  by  using 
partitions  that  are  given  by  measurement  devices.  Nonetheless,  the  models  used 
in  this  section  turn  out  to  be  identical  to  the  quantised  FPO  for  autonomous 
systems  or  to  the  set  of  quantised  Foias  operators  for  non-autonomous  systems. 
Hence,  the  results  of  the  previous  sections  show  the  connection  between  the 
qualitative  modelling  approach  and  nonlinear  systems  theory  though  the  field  of 
application  is  completely  different.  The  main  difference  is  that  the  application 
to  process  supervision  requires  completeness: 
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Definition  7.  Denote  the  variable  of  the  qualitative  model  corresponding  to 
[£c(fc)]  by  z{k).  A  qualitative  model  is  complete  if 

Prob([a;(fc)]  |po(a!),  [u(0)], [u(fc)])  >  0 
=»  Prob(z(fc)|po(a:),[w(0)],...,[u(A:)])  >  0  (17) 

holds  for  any  input  symbol  sequence  and  for  any  initial  density  Pq{x). 

4.2  Qualitative  Model  of  the  Quantised  System 

A  stochastic  automaton  <S(A4,  A/J;,  i^,p^(0))  is  used  as  qualitative  model  of  the 
system  (13) -(14).  The  set  A4  =  N}  is  the  finite  set  of  automaton  states, 

A/'v  =  {1,...,M}  the  set  of  input  symbols,  and  ^^(0)  €  the  initial  state 
probability  distribution.  The  transition  relation  F 

F  :  A4  X  A4  X  J\fy  — )■  [0,  1],  F{z'j  z,  v)  —  Prob(2:'|2:,  v) 

describes  the  conditional  probability  that  the  automaton  state  changes  from  z 
to  the  successor  state  z'  for  input  symbol  v. 

In  order  to  approximate  the  quantised  system,  the  automaton  S  is  used  with 
A4  =  A4  and  Afv  =  A/*u.  The  stochastic  automaton  defines  the  set  of  operators 

pv  .y^N  y^N  i^y 

N  N 

P>  =  with  P'’5^=Y.  •  <5"'  >  (18) 

Z=1  z'  =  l 

where  e  denotes  the  unit  vector  (0 ...  1 ...  0)  whose  z-th  element  is  equal 
to  one,  and  p^  the  z-th  component  of  the  vector  p  G  >V^. 

The  set  of  all  quantised  states  i  that  can  be  reached  by  the  quantised  system 
from  the  quantised  state  j  for  the  quantised  input  /  is  denoted  by 

TQs{jJ)  =  {i  :  Prob(2|j,/)  >  0}  (19) 

with  Prob(i|j,  1)  defined  by  eqn.  (16).  Similarly,  7f(z,  v)  denotes  the  set  of  states 
reached  from  automaton  state  z  for  input  v:  Tf(z,v)  =  {z'  :  F(z',z,u)  >  0}. 

Definition  8.  A  stochastic  automaton  S{Afz,N'y,F,p^{0))  is  called  sound  with 
respect  to  a  given  quantised  system,  if  the  following  relations  hold: 

Tf{z,v)  d  TqsU  =  z,l  =  v)  \/zeAfz,  ve  Afv  (20) 

DnPq{x)  >  0  =>  p^(0)  >  0  (componentwise)  (21) 

Theorem  3.  A  stochastic  automaton  is  a  complete  model  of  the  quantised  sys¬ 
tem  if  and  only  if  it  is  sound. 

Proof  The  necessity  is  obvious  because  if  the  automaton  is  not  sound  there  is 
at  least  one  transition  occurring  in  the  quantised  system  that  cannot  occur  in 
the  automaton,  which  violates  the  completeness  (17)  for  ^  1.  The  proof  that 
soundness  is  sufficient  is  given  in  [8] .  □ 
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According  to  this  result,  eqns.  (13)-{14)  representing  the  non-autonomous 
quantised  system  can  be  approximated  by  the  simpler  equations 

p,{k^l)  =  (22) 

Prob(2:(A:)  |  po(x),  [w(0)], [wlfc)])  =  pi  (23) 

where  the  operators  contains  a  transition  relation  F  fulfilling  condition  (20), 
and  pI  fulfils  condition  (21).  Eqns.  (22)  and  (23)  approximate  the  quantised 
system  and  fulfil  the  modelling  aim  (17). 

Remark.  The  best  sound  model  is  obtained,  if  the  transition  relation  is  set 
according  to  the  transition  probabilities  of  the  quantised  Foias  operator 


F%z\z,v)  = 


P^+^iQxu{z,v)) 


(24) 


(cf.  eqn.  (16))  and,  if  the  initial  probability  distribution  pg  =  DnPo{x)  is  used. 
F*  is  the  best  transition  relation,  i.e.  it  contains  the  smallest  possible  set  of 
transitions  necessary  for  soundness. 


5  Sound  Abstraction  of  Qualitative  Models 


Whereas  it  is  easy  to  fulfil  the  soundness  condition  (21)  it  is  difficult  to  prac¬ 
tically  compute  a  transition  relation  F  for  given  quantised  system  such  that 
condition  (20)  is  satisfied.  In  this  section  a  method  for  computing  F  is  presented 
that  guarantees  soundness  and  converges  to  F*  for  increasing  approximation 
accuracy.  Furthermore  it  is  explained  why  the  ’’classical”  point  mapping  does 
not  guarantee  that  the  resulting  models  are  sound  and,  hence,  cannot  be  used 
to  compute  qualitative  models  for  process  supervision. 

Both  methods  presented  in  this  section  use  a  ’’forward”  way  to  determine 
the  transition  probabilities  (16): 


Prob(i|  j,  1) 


p^+^({(x,tt)  €  QxujjJ)  :  f{x,u)  e  Qx{i)}) 
P^^^{Qxu{Fl)) 


The  set  described  in  the  numerator  of  this  fraction  is  the  same  as  in  eqn.  (16) 
but  it  is  described  by  using  the  transformation  f{x,u)  instead  of  its  inverse. 


5.1  Point-Based  Cell-to-Cell  Mapping 

The  classical  method  to  compute  transition  probabilities  from  one  cell  of  a  parti¬ 
tioned  space  to  another  is  point  mapping  [4].  It  is  widely  used  for  the  analysis  of 
nonlinear  dynamical  systems  and  originally  formulated  only  for  autonomous  sys¬ 
tems.  In  the  following  the  method  is  briefly  summarised  and  applied  to  compute 
an  estimate  of  the  transition  relation. 

The  main  idea  is  to  take  a  selection  of  points  of  each  region  Q.xui^j'i  0 

QxuijJ)  ==  {{xuUi),(X2,U2),..^,{Xk,Uk)}, 
with  (x,^,'ii'/^)  £  Q,xu{,j •)  0’  ^  ^  ’ 
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such  that  they  are  uniformly  distributed  over  QxuU.l)^  For  each  point 
of  Qxuij.  0  eqn.  (10)  is  used  to  compute  the  map  u^).  The  partition  of  Q 
is  used  to  determine  the  successor  state  i.  Doing  this  for  all  points  «  =  1, . . . ,  jFiT 
the  sets 


QMiJJ)  =  {{x,u)  \  {x,u)  G  Qxu{3,1)J{x,u)  G  Qx(i)},  i  =  (25) 

can  be  constructed.  Then  the  transition  relation  is  approximated  by 


A{Q.uiz,v)) 


where  yl(-)  denotes  the  number  of  points  contained  in  the  set  and  F~  denotes  the 
obtained  transition  relation.  According  to  the  law  of  large  numbers  this  estimate 
of  the  transition  probabilities  converges  to  F*  for  K  oo. 

This  method  can  directly  be  implemented  on  a  computer.  As  the  sets  Qxu(j,  1) 
are  finite  and,  therefore,  a  finite  number  of  mappings  with  eqn.  (10)  leads  to 
QxuihjJ),  all  sets  can  be  stored  in  a  computer  memory.  Furthermore  the  im¬ 
plementation  is  very  simple  as  only  partition  and  mapping  functions  are  required. 
However,  the  problem  of  point  mapping  is  that  the  soundness  of  the  obtained 
model  cannot  be  guaranteed.  More  precisely,  for  F~  the  relation 


Tf-{z,v)CTf*{z,v)  V(2:,u)  €A4  X  a/;  (26) 

rather  than  the  soundness  condition  (20)  holds.  This  means  that  only  in  the  ideal 
case  that  Ff-  (2,  v)  =  Tf*  (^,  v)  holds  for  all  2;,  a  complete  model  is  obtained. 
Practically  the  number  of  points  to  be  mapped  must  be  so  high  that  with  rea¬ 
sonable  computational  effort  even  for  simple  systems  only  an  incomplete  model 
with  Tf-  {z,  v)  c  Tf*  {z,  v)  can  be  obtained. 


5.2  Hyperbox  Cell- to- Cell  Mapping 

In  this  section  a  method  is  presented  that  guarantees  soundness.  It  is  assumed 
that  /  satisfies  a  Lipschitz  condition,  i.e.  a  number  (j)  e  IR'*'  exists  such  that 


ll(/(»i,Ui)  -  /(aJ2,W2))||oo  < 


fxi-X2'\ 

~U2  J 


00 


(27) 


holds  where  the  infinity  norm  ||  •  ||oo  is  used.  For  simplicity  reasons  it  is  as¬ 
sumed  that  all  partitions  are  orthogonal  resulting  in  hyper  boxes  and 
(cf.  Figure  4). 

The  idea  will  be  described  by  using  Figure  4,  On  the  left-hand  side  of  the 
figure  the  state-input-space  O  x  ^  is  shown.  The  dashed  lines  symbolise  the 
partition  bounds  and  the  cell  QxuU^^)  to  be  mapped,  depicted  in  light  gray,  is 
assumed  to  be  quadratic  with  a  sidelength  of  2r°.  Initially  only  the  black  centre 
point  (a3*^,it®)  is  positioned  used.  The  cell  can  be  described  by: 


Qxu{jJ)  =  {{X,u)  I  \\{x,u)  -  (ic‘^,w°)||^  <  r°}  . 


(28) 
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Fig.  4.  Hyperbox  mapping. 


The  point  is  mapped  to  the  state-space  17  by  eqn.  (10)  resulting  in 

the  black  point  /(£c°,n°)  depicted  on  the  right-hand  side  of  Figure  4  (for  s  = 
0).  Due  to  the  Lipschitz  constraint  (27)  the  map  of  the  box  QxuUJ)  can  be 
overapproximated  by  the  box 

=  {X  I  ||x  -  ■<!>}■ 

which  degenerates  to  an  interval  in  Figure  4.  If,  as  in  the  figure  the  set 
overlaps  with  more  than  one  partition  of  17,  it  is  not  known  whether  this  overlap 
results  from  the  overapproximation  of  the  map  of  Qxu-  Therefore,  the  box  in  the 
state-input-space  17  x  iP'  is  subdivided  into  =  9  boxes.  The  additional  grey 

points  are  mapped,  resulting  in  grey  intervals  in  Figure  4,  for  s  =  1  with  length 
r^(t)/3.  A  corner  check  reveals  whether  or  not  the  boxes  lie  completely  within 
one  region  i  of  the  partitioned  state  set.  Mapped  boxes  that  cover  more  than 
one  partition  region  of  17  have  to  be  further  subdivided  in  the  state-input-space. 
The  subdivision  can  be  stopped  when  the  size  of  the  mapped  approximation 
boxes  with  radius  became  smaller  than  the  partition  of  17,  where  s  is 

the  number  of  subdivisions.  Then  the  quantisation  of  the  corners  of  the  mapped 
boxes  cover  all  possible  successor  states  i  that  could  possibly  be  reached  by 
the  map  of  Qxu{jJ)’  This  guarantees  soundness.  The  subdivision  can  also  be 
continued  to  further  increase  the  accuracy  of  the  approximation.  In  Figure  4 
the  subdivision  can  be  stopped  at  s  =  1  to  guarantee  soundness.  However  it  is 
continued  to  determine  the  areas  of  Qxuih^jy^)  and  Qxu{i2^jJ)  more  precisely. 

Theorem  4.  The  hyperhox  cell-to-cell  mapping  yields  a  sound  automaton.  The 
estimate  of  the  transition  probabilities  converges  to  F*{z',  z,v)  for  all  z',  z,  v  for 
increasing  number  of  subdivisions. 

Due  to  space  limitations  the  method  cannot  be  formally  introduced  and 
proved  in  this  paper.  However,  the  soundness  becomes  clear  from  the  Lipschitz 
condition,  as  the  map  of  each  box  is  conservatively  approximated  and  the  whole 
partition  region  Qxu{jJ)  of  the  state-input  space  is  covered  by  boxes.  Figure  4 
shows  that  the  relevant  Borel  measures  in  the  state-input-space  are  approxi¬ 
mated  with  increasing  accuracy  if  more  and  more  subdivisions  of  the  boxes  are 
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used.  Simultaneously  the  estimate  of  the  transition  probabilities  converges  to 
the  corresponding  value  of  F*. 

6  Conclusions 

It  has  been  shown  that  quantised  systems  can  be  represented  by  the  Probenius- 
Perron  operator  for  autonomous  or  the  Foias  operator  for  non- autonomous  sys¬ 
tems.  As  a  consequence,  results  of  the  FPO  theory  can  be  applied  to  quantised 
systems.  It  is  shown  that  a  finite  approximation  of  the  FPO  is  identical  to  the 
abstraction  of  the  quantised  system  in  form  of  a  stochastic  automaton.  With  this 
it  has  been  shown  that  the  abstraction  converges  to  the  FPO  with  finer  quan¬ 
tisation.  Furthermore,  methods  have  been  presented  to  compute  abstractions  of 
the  quantised  system.  A  hyperbox  cell  mapping  method  has  been  presented  to 
guarantee  the  completeness  of  the  abstraction. 
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Abstract.  The  algorithmic  design  of  least  restrictive  controllers  for  hy¬ 
brid  systems  that  satisfy  reachability  specifications  has  received  much 
attention  recently.  Despite  the  importance  of  algorithmic  approaches  to 
controller  design  for  hybrid  systems,  results  that  guarantee  termination 
of  the  algorithms  have  been  limited.  In  this  paper,  we  extend  recent 
decidability  results  on  controller  synthesis  for  classes  of  linear  hybrid 
systems  to  semi-decision  procedures  for  triangular  hybrid  systems  which 
can  be  used  to  model  nonholonomic  systems  after  a  transformation.  Our 
results  are  then  applied  to  verification  of  a  conflict  resolution  maneuver 
from  air  traffic  control. 

1  Introduction 

Safety  criticality  in  motivating  applications  [13]  of  hybrid  systems  has  resulted  in 
much  research  on  computing  reachable  sets  for  hybrid  systems  in  order  to  ensure 
that  these  systems  avoid  unsafe  regions  of  the  state  space  [2,3,4].  Furthermore, 
much  research  has  recently  focused  on  controller  synthesis  of  hybrid  systems 
where  the  safety  property  is  ensured  by  design  [1,6,7,12]. 

The  complexity  of  the  motivating  applications  makes  algorithmic  approaches 
to  controller  synthesis  very  desirable,  whenever  possible.  However,  termination 
guarantees  for  algorithmic  approaches  to  synthesis  have  been  limited.  In  partic¬ 
ular,  the  game  theoretic  framework  for  controller  synthesis  introduced  in  [6]  was 
only  recently  shown  to  result  in  decision  procedures  for  various  classes  of  linear 
systems  [9],  and  semi-decision  procedures  for  classes  of  linear  hybrid  systems  [10]. 

In  this  paper,  we  proceed  along  the  same  spirit  of  [9,10]  but  we  increase  the 
complexity  of  the  continuous  dynamics  to  capture  triangular  hybrid  systems^ 
which  are  defined  as  hybrid  control  systems  whose  continuous  dynamics  in  each 
discrete  state  are  nonlinear  with  a  triangular  structure.  Triangular  nonlinear 
systems  is  a  rich  class  of  nonlinear  systems  that  capture  the  so-called  chained 
systems^  which  can  be  used  to  model  nonholonomic  systems  after  a  state  trans¬ 
formation.  Nonholonomic  systems  have  been  very  useful  kinematic  models  of 
aircraft,  robots,  space  robots,  etc  [5].  In  this  paper,  we  consider  the  follow¬ 
ing  controller  synthesis  problem:  Given  a  triangular  hybrid  system,  compute  the 
maximal  control  invariant  set  of  initial  conditions  and  least  restrictive  controller 
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such  that  for  all  disturbances  the  state  will  avoid  an  unsafe  set.  In  particular, 
we  present  a  semi-decision  procedure  which,  if  it  terminates,  exactly  solves  the 
above  problem. 

The  solution  of  the  above  problem  depends  critically  on  state  of  the  art 
techniques  from  controller  synthesis  of  hybrid  systems.  In  particular,  we  adopt 
the  general  framework  for  controller  s5mthesis  of  nonlinear  hybrid  systems  [6], 
while  we  follow  in  spirit  the  approach  taken  in  [9].  In  particular,  we  focus  on 
continuous  games  for  triangular  nonlinear  systems.  Application  of  the  maximum 
principle  leads  to  bang-bang  optimal  controls  and  a  triangular  structure  in  the 
co-state  equations.  Rather  than  solving  the  Hamilton- Jacobi  partial  differential 
equations  for  reachability  computations,  we  abstract  the  bang-bang  nature  of  the 
optimal  control  to  a  hybrid  system.  The  piece- wise  constant  nature  of  the  optimal 
inputs  and  disturbances,  and  the  triangular  structure  of  the  state  and  co-state 
dynamics  leads  to  polynomial  flows  for  the  states  and  co-states.  This  allows  us  to 
use  quantifier  elimination  in  each  discrete  state  of  the  abstracted  game  to  perform 
reachability  computations.  The  above  sequence  of  steps  results  in  a  semi-decision 
procedure  for  controller  synthesis  for  triangular  hybrid  systems.  However,  unlike 
classes  of  linear  systems  where  the  number  of  switchings  is  uniformly  finite  [9], 
no  such  guarantee  exists  for  triangular  systems,  making  very  difficult  any  claims 
for  a  decision  procedure. 

The  structure  of  this  paper  is  as  follows:  In  Section  2  we  review  the  synthesis 
framework  of  [6],  In  Section  3  we  present  a  semi-decision  procedure  for  reach 
set  computation  in  triangular  nonlinear  systems,  which  is  lifted  in  Section  4 
to  triangular  hybrid  systems.  These  results  are  then  applied  in  Section  5  to  a 
verification  of  a  conflict  resolution  maneuver  from  air  traffic  control. 

2  Controller  Synthesis  for  Nonlinear  Hybrid  Systems 

In  this  section  we  review  the  framework  for  computing  the  maximum  controlled 
invariant  safe  set  for  general  nonlinear  hybrid  systems  [6,12]. 

Definition  1  (Hybrid  system). 

A  hybrid  system  H  is  a  collection  (A,  V, /,/,£;,  ^),  with: 

-  State  and  input  variables:  X  and  V  are  disjoint  collections  of  state  and 
input  variables.  We  assume  that  X  =  Xd^Xc  and  V  =  Vd^Vq,  where  Xc 
and  Vc  contain  continuous,  and  Xd  and  Vjj  discrete  variables.  We  refer  to 
valuations  a:  €  X  and  v  G\’  as  the  state  and  the  input  of  the  hybrid  system. 

-  Initial  states:  I  CX  is  a  set  of  initial  valuations  of  the  state  variables. 

-  Continuous  evolution:  /  :  X  x  V  ^  TXc  is  a  vector  field. 

-  Discrete  transitions:  E  C  X  x  V  x  X  is  a  set  of  discrete  transitions. 

-  Admissible  inputs:  (j)  :  X  2^  gives  the  set  of  admissible  inputs  at  a 
given  state  x  gX. 

It  is  customary  to  use  the  notation  {q,x)  ~  {^\xd^^\xc)  ^  The  meaning  of 
the  variable  x  will  be  clear  from  the  context. 
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For  any  input  v  =  {u^d)  G  V,  define  the  set: 

Inv{v)  =  {a:  G  X  I  G  (j){x)  A  {x^v^x)  G  E}. 

For  a  state  x  G  X  and  input  v  =  {u,d),  define: 

Next{x,v)  = 

Inv{v)  is  the  set  of  states  from  which  continuous  evolution  is  possible  under 
input  V,  while  Next{x,  v)  is  the  set  of  states  that  can  be  reached  from  x  under 
input  V  through  a  discrete  transition.  For  any  set  K  C  X  and  input  v  =  (u,  d) 
the  successor  of  K  under  v  is  given  by  Next{K,v)  =  Elextyx^v). 

For  any  set  K  C  X  define  the  controllable  predecessor  of  K,  Preu(K),  and 
the  uncontrollable  predecessor  of  Pred{K),  by: 

Preu{K)  ^  {x  eX\3u  eV  W  eU  X  ^  Inv{v)  A  Next{K,  v)CK}r\  K, 
Pred{K)  ^  {x  G  X  I  Vu  G  U  3d  G  D  Next{K,  v)nK^  K\ 

where  v  —  (u,d).  Pre^iK)  contains  all  states  in  K  for  which  u  can  force  a 
transition  back  into  K.  Pred{K)  contains  all  states  outside  K  together  with 
those  states  for  which  it  is  possible  to  transition  outside  K  regardless  of  the 
action  of  u.  Whereas  Preu  and  Pred  capture  information  about  regions  of  the 
state  space  that  can  be  reached  through  discrete  transitions  of  the  system,  the 
following  operator  [12]  captures  continuous  reachability  information. 

Definition  2  (Reach- Avoid).  Given  a  hybrid  system  H  and  disjoint  sets 
G  C  X,  the  operator  Reach  :  2^  x  2^  2^  is  defined  as: 

Reach{K,  G)  =  {xq  \  Vu  G  U3d  eV3t>0:  x{t)  G  iT  A  Vs  G  [0,  t]  x(s)  ^  G}, 

were  U,  V  denote  the  set  of  piecewise  continuous  functions  from  the  IR  to  U,  D 
respectively,  and  x(-)  is  the  unique  state  trajectory  starting  from  initial  condition 
x(0)  =  Xq  under  the  input  (u,d). 

The  set  Reach{K,  G)  contains  the  states  from  which  for  all  controls  there  exists 
a  disturbance  such  that  the  state  trajectory  can  be  driven  to  K  while  avoiding 
the  escape  set  G.  The  following  algorithm  uses  the  Reach  operator  to  compute 
the  maximal  controlled  invariant  subset  of  F  (see  [12]). 

Algorithm  1  (Maximum  Controlled  Invariant  Safe  Set) 
initialize 

=  F;  W-i  =  0;  i  =  0 
while  ^ 

=W^\  Reach{Pred{W^),  Pre^(W')) 
i  —  i  —  1 

end  while 
W*  := 
end 


f  {t/  G  X  I  (x,  V,  y)eE}  if  i;  G  (/)(x) 
^  0  ii  V  ^ 
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Algorithm  1  iteratively  removes  from  the  safe  set  F  all  states  for  which 
there  is  a  disturbance  which  either  through  continuous  evolution  or  discrete 
transition  can  bring  the  system  outside  F  regardless  of  the  control  action.  In 
order  to  implement  Algorithm  1,  one  needs  to  encode  sets  of  states,  perform 
set  intersection,  union,  test  for  emptiness,  and  exactly  compute  Reach{‘,  •).  If  all 
these  conditions  hold  for  a  class  of  systems,  then  the  problem  is  semi- decidable 
for  that  class  of  systems.  Even  though  there  is  no  guarantee  of  termination,  if  the 
algorithm  terminates,  then  it  exactly  computes  the  unique  maximal  controlled 
invariant  set  W*.  If  in  addition.  Algorithm  1  is  guaranteed  to  terminate  after 
a  finite  number  of  iterations  for  a  cleiss  of  systems,  then  we  say  the  problem  is 
decidable  for  that  class. 

The  main  difficulty  in  the  implementation  of  Algorithm  1  is  the  computation 
of  the  Reach  operator.  For  general  nonlinear  hybrid  systems,  the  computation 
of  Reach  relies  on  the  numerical  solution  of  a  pair  of  coupled  Hamilton- Jacobi 
partial  differential  equations  [7,12].  In  this  paper,  we  show  that  for  a  certain 
class  of  nonlinear  hybrid  systems  with  triangular  continuous  dynamics  each  step 
of  Algorithm  1  is  symbolically  computable.  This  class  is  rich  enough  to  capture 
hybrid  systems  with  chained  nonlinear  dynamics,  which  model  nonholonomic 
kinematics  for  aircraft,  cars,  and  robots. 

3  Computing  Safe  Sets  for  Triangular  Nonlinear  Systems 

In  this  section,  we  address  the  problem  of  computing  maximal  controlled  invari¬ 
ant  safe  sets  for  a  class  of  nonlinear  control  systems  subject  to  disturbances.  The 
computation  of  maximal  safe  sets  is  a  fundamental  step  in  the  least  restrictive 
controller  synthesis  problem  [6].  In  this  section,  we  extend  the  methodology  of 
symbolic  controller  synthesis  for  classes  linear  systems  described  in  [9]  to  a  class 
of  nonlinear  systems. 

For  a  differential  game  x  =  f{x,  u,  d)  between  inputs  ueU  c  IR”“  and  dis¬ 
turbances  d  Q  D  C  ,  the  solution  to  the  controller  synthesis  problem  requires 
the  computation  of  the  set  of  initial  states  for  which  there  exists  a  disturbance 
that  can  eventually  drive  the  system  to  some  unsafe  set  regardless  of  the  ac¬ 
tions  of  the  control.  Therefore  the  controller  synthesis  problem  for  continuous 
time  system  requires  the  computation  of  the  continuous  system  version  of  the 
Reach- Avoid  set. 

Definition  3  (Reach- Avoid).  Given  a  differential  game  x  =  f{x,u,d)  and 
disjoint  sets  K,G  the  operator  Reach  :  2^"  x  2^"  2*^"  is  defined  as: 

Reach{K,  G)  ^  {xq  e  U  3d  ^  V  3t  >  0  :  x{t)  G  K  A\/s  e  [0,  t]  x{s)  ^  G}, 

where  lA ,  D  denote  the  set  of  piecewise  continuous  functions  from  the  ]R  to  U,  D 
respectively,  and  x(^-^  is  the  unique  state  trajectory  of  x  —  fix,u,d)  starting  from 
initial  condition  2:(0)  =  xq  under  the  input  (u,  d). 

The  set  Reach{K,G),  which  is  graphically  depicted  in  Figure  1,  contains  the 
states  from  which  for  all  controls  there  exists  a  disturbance  such  that  the  state 
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Fig.  1.  Showing  a  graphical  depiction  of  Reach{K,G). 


trajectory  can  be  driven  to  K  while  avoiding  the  escape  set  G,  It  was  shown 
recently  that  the  computation  of  Reach  is  decidable  for  certain  classes  of  linear 
systems  [10].  Here  we  extend  the  result  to  a  class  of  nonlinear  systems.  As  a 
motivating  example,  consider  the  following  nonlinear  system  in  so-called  chain 
form: 

j  =  1, . . .  ,  m 

j  =  1, . . .  ,  m  and  i  <  j  (1) 

iy  =  x^~^Uj  j  =  1,...  , m  and  i  <  j  and  k  =  2,.. .  ,nj. 

Control  systems  of  the  class  shown  in  equation  (1)  are  quite  important  because 
they  can  be  used  to  model  many  types  of  nonholonomic  and  under-actuated 
systems  including  unicycles,  cars,  multi-steering  trucks  with  iV-trailers,  space 
robots,  etc.  [8].  We  now  apply  the  symbolic  controller  synthesis  methodology 
described  in  [9,10]  to  this  chain  form  system. 


3.1  Computation  of  Optimal  Control 

For  the  chain  form  system  (1),  suppose  we  wish  to  compute  the  set  of  initial 
conditions  W  C  IR^  for  which  there  exists  a  control  u(-),  constrained  to  a  com¬ 
pact  rectangular  feasible  control  set  U  C  that  can  steer  the  state  to  the 
goal  G  C  while  avoiding  states  B  c  IR’^.  This  problem  is  closely  related  to 
the  problem  of  nonholonomic  motion  planning  in  the  presence  of  obstacles  [5] 
and  is  equivalent  to  computing  W  =  Reach{G,B). 

To  solve  the  reachability  problem,  we  first  introduce  the  co-state  p  e  and 
construct  the  Hamiltonian: 


H{x,p,u)  =  p^f{x,u)  =  (p°  +  ELi  (Pij^i  +  '))  “j- 
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The  Hamiltonian  satisfies  the  state  and  co-state  differential  equations  x  = 
^ ,  p  =  —  ^ .  Prom  the  Hamiltonian,  we  compute  the  co-state  dynamics; 

p”/  =0  j  =  1, . . .  ,  m  and  i  <  j 

=  —PijUj  jf  =  1, . . .  ,  m  and  i  <  j  and  k  =  2,,,,  ,nj 

•0  \-^7n  1  •  i 

Pi  J  = 

Notice  that  the  chain  structure  of  the  system  dynamics  is  inherited  by  the  co¬ 
state  dynamics.  Next,  we  initialize  the  co-state  as  the  inward-pointing  normal 
on  the  boundary  of  G  and  apply  the  Pontryagin  Maximum  Principle  to  compute 
the  optimal  control  u*  =  argmax^gt;  i/(x,p,  u).  Since  the  feasible  control  set  is  a 
compact  rectangle  U  =  j]  C  we  may  decompose  the  Maximum 

Principle  for  each  component  of  the  input: 


u. 


arg  max 


U^. 


(2) 


3.2  Construction  of  Hybrid  System 

The  Maximum  Principle  calls  for  bang-bang  controls:  the  optimal  controls  will 
always  lie  on  the  vertices  on  the  feasible  control  set  U.  From  equation  (2),  it  is 
direct  to  see  that  u*  is  either  Uj  or  Uj  depending  on  the  sign  of  the  “switching 
function”  of  the  state  and  co-state  which  multiplies  Uj.  Thus,  as  proposed  in  [9, 
10]  we  can  construct  a  hybrid  system  which  has  2^  -h  1  discrete  states;  One 
discrete  state  for  each  vertex  of  the  rectangle  U,  and  one  discrete  state  for  stop¬ 
ping  the  reachability  computation  on  the  obstacle  set  B  (see  [10]).  The  guards 
and  invariants  for  the  constructed  hybrid  system  are  defined  by  the  “switching 
functions”  in  the  optimal  control  shown  in  equation  (2). 


3.3  Reach  Set  Computation 

For  each  discrete  state  of  the  constructed  hybrid  system  we  need  to  solve  a 
reachability  computation  for  a  system  of  the  form: 


II 

O-r., 

.  ,  m 

ilj  =  x°u* 

3  =  1,- 

.  ,  m  and  i  <  j 

i  =  1,.. 

.  ,  m  and  i  <  j  and  k  = 

2,,. 

.  ,nj 

II 

o 

3  =  h- 

.  ,  m  and  i  <  j. 

p’ij  ^  =  -Pij^j 

j  =  T  •  • 

.  ,  m  and  i  <  j  and  k  = 

2,.. 

.  ,nj 

2  =  1,.. 

.  ,m. 

where  is  a  constant  rational  number.  It  is  easily  shown  that  the  problem  of 
computing  the  reachable  set  of  this  system  is  decidable.  Indeed,  due  to  the  chain 
form  of  the  state  and  co-state  dynamics,  we  may  iteratively  compute  the  fiow 
of  the  system  by  symbolic  integration  and  substitution  starting  from  and 
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proceeding  down  the  chain.  By  symbolic  integration  the  flow  of  this  system  is 
computed  to  be: 


x^{t)  =x^{0)  +  u*t  i  = 


Ihjit) 

=  p5’'(o) 

j  =  1,... 

,  m  and  i  <  j 

pU^) 

j  =  U-*- 

,  m  and  i  <  j  and  k  = 

pm 

=p?(o)+Erir‘^pi^(o) 

i  =  1,... 

,  m. 

We  use  the  notation  x{t)  =  (t){xo,  u,  t)  to  denote  the  state  x{t)  which  is  a  result 
of  flowing  for  t  seconds  along  the  dynamics  of  the  system  with  input  u  starting 
at  the  initial  condition  a:(0)  =  2:0.  Since  the  flow  of  this  system  is  polynomial,  it 
admits  quantifier  elimination  [11],  and  hence  the  computation  of  the  set  of  points 
which  can  reach  a  semi-algebraic  set  K,  {xo  €  |  >  0  :  (l)(xo,Upt)  G  K} 

for  each  discrete  state  of  the  constructed  hybrid  system  is  decidable. 

The  only  remaining  condition  of  interest  for  the  constructed  hybrid  system 
is  an  upper  bound  on  the  number  of  switchings  between  the  discrete  states. 
For  the  case  of  linear  systems  with  dynamic  matrices  that  are  either  nilpotent  or 
diagonalizable  with  real  rational  eigenvalues,  a  result  of  Pontryagin  provides  that 
the  number  of  switchings  of  the  optimal  control  is  no  greater  than  the  dimension 
of  the  system.  For  these  classes  of  systems,  we  are  able  to  show  decidability  of 
the  least  restrictive  controller  synthesis  problem  [9].  We  can  make  no  such  claim 
in  the  case  of  chain  form  systems  of  the  type  in  equation  (1).  In  general  there 
is  no  upper  bound  on  the  number  of  switchings  on  the  optimal  control  defined 
in  (2).  Hence  we  conclude  that  controller  synthesis  problem  for  the  class  of  chain 
form  systems  is  semi' decidable. 


3.4  Triangular  Systems 

Upon  examination,  we  realize  that  there  are  essentially  two  features  in  the  struc¬ 
ture  of  chain  form  systems  that  allow  the  above  methodology  to  work: 

1.  The  vector  field  has  linear  terms  in  u. 

-  Thus  the  Hamiltonian  has  linear  terms  in  u,  and  applying  the  Maximum 
Principle,  we  see  that  the  optimal  input  u*  is  piecewise  constant  on  the 
vertices  of  the  feasible  control  set. 

—  This  allows  us  to  construct  a  hybrid  system  out  of  the  switching  logic  of 
the  optimal  control,  where  for  each  discrete  state  there  is  a  constant  u*. 

2.  The  time  derivative  of  each  state  is  a  polynomial  in  the  input  and  the  pre¬ 
ceding  states  of  the  chain. 

-  For  a  constant  u*  the  flow  can  be  computed  iteratively  by  symbolic 
integration  and  substitution  starting  from  the  beginning  of  the  chain. 

—  Since  u*  is  constant  and  the  vector  field  depends  polynomially  in  states, 
the  flow  of  the  system  is  polynomial  in  n*,  t  and  the  state. 
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This  structure  is  inherited  by  the  co-state  dynamics  and  hence  the  flow 
of  the  co-state  can  also  be  symbolically  integrated. 

The  observation  above  suggests  that  the  methodology  for  symbolic  reach¬ 
ability  computation  will  also  work  on  the  following  larger  class  of  triangular 
nonlinear  systems. 

Definition  4  (Triangular  nonlinear  system). 

A  nonlinear  system  x  =  f[x,u)  is  called  triangular  if  it  can  be  written  as: 

Xo=a  +  jyjli 
±1  ~  flix^)  + 

X2  =  f2{Xo,x{)  +  YJ^^i923{X0,Xi)Uj 

fn{Xo,  •  •  •  ,  Xn—l)  -\-  =  l  9nj{x0')  •  •  •  5  Xn—l)Uj^ 

where  a,  bj  G  Q  and  fi,  Qij  G  Q[a;o,  -  ■ .  ,  for  z  =  1, . . .  ,  n  and  j  =  1, . . .  ,  m. 

Moreover,  it  is  direct  to  see  that  the  methodology  is  also  applicable  to  the  class  of 
triangular  differential  games  between  inputs  u  G  and  disturbances  d  G 


Definition  5  (Triangular  differential  game). 

A  differential  game  x  —  f(x^u,d)  is  called  triangular  if  it  can  be  written  as: 

XQj  =  Qj  -f-  ^jk'^k  4*  Y2k=l  ^jkdk 

flj{xoi:  •  .  .  ,  Xqi,)  gijk{xoi^  •  •  ■  )  Xoi,)Uj  -j- 

^A:=l  ^ljk(^k{xoi,  .  .  .  jXqi) 

^ij  ~  fij  {xoi^  ■  •  •  ,  Xqi,,  .  .  .  ,  -1- 

^k=l  9ijk{Xoi,  ‘  ,  X^i_i'^L)Uj  -f 

Ylk^i^ijk{xoi^  ‘  •  ,xoL,... 

for  j  =  1,...  ,L,  and  i  =  1,...  ,nj,  and  where  Oj,  bjk,  Cjk  G  Q  and 
fiji  9ijk^  hijk  are  polynomials  with  rational  coefficients. 

Theorem  1  (Semi-decidable  reach  for  triangular  diflferential  games). 

For  a  triangular  differential  game  x  =  /(x,  if  the  inputs  and  disturbances 
are  constrained  to  compact  rectangles  with  rational  coefficients,  then  for  any 
disjoint  semi- algebraic  sets  K,G  C  IR^,  the  problem  of  computing  Reach{K,G) 
is  semi-decidable. 

Proof  We  need  to  show  that  the  methodology  for  symbolic  reach  set  computa¬ 
tion  proposed  in  [9,10]  can  be  applied  to  triangular  differential  games  and  that 
each  step  in  the  methodology  is  computable. 
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1.  Compute  Optimal  Control.  Since  the  vector  field  can  be  written  as  i;  = 
fi{xyu)  +  f2{x,d),  the  Hamiltonian  H  =  f{x,u,d)  is  separable,  which 
implies  that  there  exists  a  saddle  solution  {u*,d*)  of  optimal  control  and 
disturbance: 

u*  =  argmaxp^/i(a:,u),  d*  =  sxgmiup'^  f2{x,d).  (4) 

uGU  d^D 

Moreover,  since  the  Hamiltonian  has  linear  terms  in  u  and  d,  and  the 
sets  of  feasible  controls  and  disturbances  are  compact  rectangles  U  = 
^  =  IYi=i[S.j,Dj]  C  IR'"^  we  may  decompose  equa¬ 
tion  (4)  to  get: 

u*  —  arg  max  s‘^{x,p)  Uj,  d*  =  arg  max_  s^{x,p)  dj,  (5) 

^  Uj€\U.,Uj]  dje\D.,Dj] 

where  s^(')  and  s^(-)  are  “switching  functions”  which  are  polynomial  in 
the  state  and  co-state  {x,p).  The  Maximum  Principle  calls  for  bang-bang 
optimal  controls  and  disturbances:  Depending  on  the  signs  of  the  switching 
functions,  the  optimal  controls  and  disturbances  will  always  lie  on  a  vertex 
of  the  feasible  control  and  disturbance  set. 

2.  Construct  Hybrid  System.  Construct  a  hybrid  system  with  2^^  discrete 

states  for  each  possible  optimal  control,  2'^^  discrete  states  for  each  possible 
disturbance,  and  one  discrete  state  for  stopping  the  reachability  computation 
on  the  avoid  set  G  (see  [10]).  The  switching  functions  s^(*)  determine 

the  discrete  transitions  of  the  constructed  hybrid  system,  and  continuous 
dynamics  are  the  co-state  dynamics  p  =  —  ^  appended  to  i:  =  f{x,u*,d*) 
where  {u*,d*)  are  constant. 

3.  Calculate  Reach  Set.  In  each  discrete  state,  the  triangular  structure  of  the 
state  dynamics  and  the  fact  that  the  optimal  control  and  disturbance  {u*,  d*) 
are  constant  allows  the  flow  of  the  state  dynamics  to  can  be  computed  by 
symbolic  integration.  Moreover,  it  is  direct  to  check  that  the  co-state  dy¬ 
namics  inherit  the  triangular  structure  of  the  state  dynamics  and  that  the 
flow  of  the  co-state  dynamics  can  also  be  integrated  symbolically.  Since  the 
flow  in  each  discrete  state  of  the  constructed  hybrid  system  is  polynomial, 
we  may  perform  quantifier  elimination  to  compute  the  reachable  set  for  each 
discrete  state  of  the  hybrid  system. 

We  have  constructed  a  hybrid  system  for  which  the  problem  of  computing  the 
reach  set  of  each  discrete  state  is  decidable.  By  initializing  the  hybrid  system  with 
the  usable  part  of  the  unsafe  set  K  (see  [9]),  we  have  a  semi-decision  procedure 
for  computing  Reach{K,G).  However,  since  in  general  there  is  no  bound  on  the 
number  of  times  the  switching  functions  change  sign,  there  is  no  bound  on  the 
number  of  discrete  transitions  the  hybrid  system  takes,  and  hence  we  cannot 
guarantee  that  the  reach  set  computation  will  terminate.  □ 
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4  Controller  Synthesis  for  Triangular  Hybrid  Systems 

The  results  of  the  previous  section  naturally  inspire  the  following  definition. 
Definition  6  (Triangular  hybrid  system). 

A  hybrid  system  H  =  {X^  V,  /,  /,  E,  (j))  is  called  a  triangular  hybrid  system  if 
Vg  e  Xd  the  set  of  feasible  inputs  (j){q^x)\vc  —  Ug  x  Dg,  where  Ug  and  Dg 
are  compact  rectangles  with  rational  vertices,  the  reset  relation  £*  C  X  x  V  x  X 
is  semi-algebraic,  and  for  each  discrete  state  q  the  vector  field  f{q,x,u,d)  is 
triangular  with  rational  coefficients. 

The  results  of  the  previous  section  provide  that  for  each  discrete  state  of  the 
hybrid  system,  the  computation  of  Reach  is  semi-decidable.  Hence  if  the  discrete 
transition  Prcd  and  PrCu  are  computable  (they  are  when  the  reset  relation  E  C 
X  X  V  X  X  is  semi-algebraic),  then  each  iteration  of  Algorithm  1  is  computable, 
and  hence  we  conclude  that  the  problem  of  computing  the  maximum  controlled 
invariant  set  is  semi-decidable. 

Theorem  2  (Semi-decidable  controller  synthesis  for  triangular  hybrid 
systems).  For  a  triangular  hybrid  system  H  and  a  semi-algebraic  safe  set  F, 
the  problem  of  computing  the  maximum  controlled  invariant  set  W*  C  F  is 
semi-decidable. 


If  the  computation  of  maximal  safe  set  W*  terminates,  we  would  like  to 
provide  a  least  restrictive  controller  that  renders  W*  invariant.  Since  the  con¬ 
tinuous  dynamics  of  triangular  hybrid  systems  are  polynomial,  the  definition  of 
the  least  restrictive  controller  can  be  written  as  a  quantified  first  order  formula 
in  the  theory  of  reals.  Hence  the  least  restrictive  controller  can  be  computed  by 
quantifier  elimination  and  is  given  in  the  following  proposition  [10]. 

Proposition  1  (Least  restrictive  controller).  Given  a  triangular  hybrid  sys¬ 
tem  H  and  a  semi-algebraic  maximal  controlled  invariant  set 

=  {x  6  ]R"  I  Vf.j  (A^ii  h,,  (x)  <  o)  }  , 

the  least  restrictive  controller  g[x)  :  X  — )•  2^  that  renders  W*  invariant  is 
computable  and  is  given  by: 


9{x)  = 


'  {«  G  4>{^)\u  I  Vd  G  <t>{x)\D  :  Next{x,  (u,d))  C  W*}  if  a:  G  (W*)° 

{u  G  d>{x)\u  I  lVf=i(Afcii(^i.  W  =  0)  =>  Vd  G  4>{x)\d  : 

'  («, d))  <  0)  Ax  G d)]V 

[Vd  e  (t>{x)\D  :  Next{x,  {u,d))  CW*  Ax  f  Inv(u,  d)]},  if  a:  e  dW* 

.(t>{x)\u.  ifxe{W*y. 


Triangular  hybrid  systems  is  the  first  known  class  of  nonlinear  hybrid  sys¬ 
tems  which  has  a  semi-decidable  controller  synthesis  problem.  In  the  following 
section  we  apply  our  methodology  to  a  conflict  resolution  example  from  air  traffic 
control. 
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5  Conflict  Resolution  Example 

In  this  section  we  present  an  application  of  our  methodology  towards  verification 
of  maneuvers  for  multi-agent  hybrid  systems.  As  an  example  application  we 
verify  a  conflict  resolution  maneuver  for  air  traffic  control  similar  to  the  one 
described  in  [13],  Consider  the  following  conflict  resolution  maneuver  for  two 
aircraft: 

1.  Cruise  until  aircraft  are  ai  miles  apart; 

2.  Change  heading  by  Acf);  fly  until  lateral  displacement  of  d  miles  achieved; 

3.  Change  to  original  heading;  fly  until  aircraft  are  ct2  miles  apart; 

4.  Change  heading  by  fly  until  lateral  displacement  of  —d  miles  achieved; 

5.  Change  to  original  heading. 


<  oi 

-X 

/  Dynamics  N. 

VyCOsi^  -  ^l)  \ 

U  =  0  j 

\  Invariant  / 

<'=0 

/  Dynamics  X 

fx  ~  —Vi  +  -  ^t)  \ 

ly-  t?2sin(<^2  -  ^)  j 

\  Invariant  / 

\  OA  I 

a:*  +  r  > 


t<0 


t  < 


t> 


vi  +  Vy  sm(A^] 
2d 


•f 


Vj  +1)2  sm(A^) 


A<^ 


RIGHT 
Dynamics 
'i  —  —Vi  +  —  <f>l) 

y  “ 

Invariant 
t  >  0 


X  —X 

t/  =  y 

<l>i  ^  <f>i 


'STRAIGHT 
Dynamics 

•vi  +n2Cos(<;&2  -  <h) 

In\’ariant 
+  <  02 


Fig.  2.  Hybrid  system  model  of  aircraft  conflict  resolution  maneuver. 


The  hybrid  automaton  modeling  this  maneuver  has  discrete  states  {CRUISE, 
LEFT,  STRAIGHT,  RIGHT}  and  is  depicted  in  Figure  2.  The  continuous  dy¬ 
namics  in  each  discrete  state  is  the  relative  flow  of  the  aircraft  given  a  fixed  veloc¬ 
ity  and  heading,  {vi  is  the  velocity  and  <1)1  is  the  heading  of  aircraft  i).  The  aircraft 
are  considered  to  be  at  a  safe  distance  if  they  are  at  least  5  miles  apart.  In  the 
relative  coordinate  frame,  the  unsafe  set  is  given  by  {(re,?/)  G  |  4- 2/^  <  5}. 
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Aircraft  1  is  assumed  to  fly  at  a  fixed  velocity  Vi  and  heading  (t)i ,  while  aircraft 
2  can  switch  “modes”  and  rotate  left  or  right  a  fixed  angle  of  ±A(j).  It  is  clear 
that  the  hybrid  automaton  modeling  the  conflict  resolution  maneuver  belongs 
to  the  class  of  triangular  hybrid  systems  described  in  the  previous  sections. 

Using  the  quantifier  elimination  package  of  Mathematica  4.0,  we  computed 
the  minimal  unsafe  sets  for  each  discrete  state  of  the  automaton  for  the  scenario 
where  two  aircraft  are  approaching  each  other  with  velocities  vi  =  A,  V2  ~  5, 
with  initial  heading  difference  of  (/)2  —  and  aircraft  2  allowed  to  change 

directions  at  an  angle  of  ±A(j)  such  that  sin(±Z\(/))  =  Equations  (6)-(8)  show 
the  results  of  the  computation. 

Vi  =  4;i;2  =  5;  A  =  0 

unsaf  eCruise  =  Resolve  >  0  A  (a:  -  vit  +  Xv2ty  +  (y  +  W-  \‘^V2t)'^  <  25] 

^  -  f<®<x/3T-f)v 

(s'  =  ^  -^25  -2/2<®<\/4T-f)v 

{rM  ^  S'  <  5  A  -y'25-y^  <x<  v'25  -  V 
< S' <  <x<V4i-f) 

IJI  =  4;i'2  —  5;  A  =  I 

imsaf eLeft  =  Resolve  >  0  A  (a;  -  Vit  +  Xv2t)^  +  (?/  +  >/l  -  X‘^V2tf  <  25] 

=  (s' <  -  ^ ^  ^ ^  -  f )  V 

(s'  =  -%/25-y2  <  X  <  5^  -  l)  V 

(^  <y<5A  -v'25-j/2  <  X  <  y'25  -  y'^'j  V 
(~^  <y  <  A  -  y'25  -  <  X  < 

Vi  =  4;?;2  =  5;  A  =  -| 

unsafeRight  =  Resolve  [3<  >  0  A  (ar  -  vit  +  Xv2t)'^  +  {y  +  >/l  -  X^V2t)^  <  25] 

=  (s'  <  A 

(s'=-7y5’A-5^-^<x<5fI_a)v 
(s'  =  A  -v''2r=^  <  X  <  V 

(7^^  <  y  <  5  A -y'25 -y2  <  x  <  y'25  -  y^J  V 
(-7V^  <y<  7i/i’  A  -y/25^  <  X  <  5^  -  a) 


Since  the  relative  heading  and  velocity  of  the  two  aircraft  is  same  for  the 
CRUISE  and  STRAIGHT  flight  modes,  then  unsaf eCruise=unsafeStraight. 
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The  result  of  the  symbolic  computation  of  the  minimal  unsafe  sets  is  shown  in 
Figure  3.  The  set  unsaf  eCruise\unsaf  eLef  t  contains  the  set  of  states  which  are 
made  safe  by  the  aircraft  turning  left,  and  the  set  unsaf  eCrui se  \  unsaf  eRight 
contains  the  set  of  states  which  are  made  safe  by  the  aircraft  turning  right. 
The  set  unsaf  eCruise  \  (unsaf  eLef  t  U  unsaf  eRight)  contains  the  states  which 
are  made  safe  by  turning  either  left  or  right,  and  the  set  unsaf eCruise  n 
unsaf  eLeft  n  unsaf  eRight  shown  in  Figure  3(d)  is  the  set  of  states  which  is 
unsafe  regardless  of  the  action  the  aircraft  takes. 


Y 


(d)  unsafeCruise  A  xmsaf  eLeft  A  unsaf  eRight 


Fig.  3.  Showing  minimal  unsafe  sets  for  each  discrete  state  of  maneuver  automaton. 


6  Conclusion 

In  this  paper,  we  have  presented  the  first  class  of  nonlinear  hybrid  systems  with 
a  semi-decidable  controller  synthesis  problem.  This  class  of  triangular  hybrid 
systems  is  rich  enough  to  capture  hybrid  models  that  include  kinematic  models 
of  aircraft,  robots,  and  cars.  Our  results  were  illustrated  on  a  conflict  resolution 
example  from  air  traffic  control. 
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Abstract.  In  this  paper  we  consider  the  problem  of  extracting  an  ab¬ 
straction  from  a  hybrid  control  system  while  preserving  timed  languages. 
Such  consistent  abstractions  are  clearly  useful  as  the  abstracted,  higher 
level  model  could  be  used  for  controller  synthesis  or  verification  of  the 
more  complicated  lower  level  model.  The  class  of  abstracting  maps  we 
consider  in  this  paper  compress  only  the  continuous  states  without  aggre¬ 
gating  any  discrete  states.  Given  such  an  abstracting  map,  we  determine 
natural  conditions  that  determine  when  trajectories  of  the  original  hybrid 
system  can  be  generated  by  the  abstracted  hybrid  system.  Conversely, 
we  determine  conditions  under  which  the  two  hybrid  systems  generate 
exactly  the  same  timed  language. 


1  Introduction 

The  analysis  and  synthesis  of  hybrid  control  systems  has  received  tremendous 
attention  recently.  The  scale  of  the  motivating  applications,  such  as  air  traf¬ 
fic  management  systems  [15]  or  automotive  engine  control  systems  [4],  require 
that  the  resulting  analysis  and  control  methodologies  scale  up  efficiently,  in  or¬ 
der  to  facilitate  the  realistic  application  of  computational  methods  to  real-scale 
examples. 

One  of  the  fundamental  approaches  to  reducing  the  complexity  of  large  scale 
system  analysis  and  design  is  the  process  of  abstraction.  Prom  an  analysis  per¬ 
spective,  given  a  model  and  a  property  of  interest,  one  tries  to  extract  a  simpler 
model,  an  abstraction,  that  preserves  the  property  of  interest  while  ignoring  ir¬ 
relevant  details.  This  approach  has  been  used  successfully  in  extracting  discrete 
abstractions  of  hybrid  systems  while  preserving  many  properties  that  can  be 
expressed  in  various  temporal  logics  [3] . 

From  a  design  perspective,  given  a  hybrid  control  system,  one  would  like 
to  extract  an  abstracted  hybrid  system,  perform  the  design  at  the  higher  level 
abstraction,  and  then  refine  the  design  at  the  lower  level.  In  this  hierarchical 
setting,  a  methodology  which  extracts  a  hierarchy  of  hybrid  system  models  at 
various  levels  of  abstraction  is  critical. 

Due  to  the  complexity  of  combinatorial  problems,  the  notion  of  abstraction  is 
more  mature  in  theoretical  computer  science  than  control  theory.  For  purely  dis¬ 
crete  systems,  the  notions  of  language  equivalence,  simulation,  and  bisimulation 
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are  established  [10].  For  purely  continuous  systems,  however,  these  concepts  are 
only  recently  beginning  to  emerge.  In  particular,  in  [12],  a  notion  of  abstraction 
for  continuous  systems  was  formalized.  In  [11]  reachability  preserving  abstrac¬ 
tions  of  continuous  linear  systems  were  characterized,  leading  to  hierarchical 
reachability  algorithms  for  linear  control  systems.  In  [13],  these  results  where 
generalized  for  nonlinear  analytic  systems.  A  general  theory  of  abstraction  for 
hybrid  systems  will  clearly  merge  the  continuous  and  discrete  approaches. 

In  this  paper,  we  address  the  problem  of  extracting  a  hybrid  abstraction  from 
a  hybrid  control  model  while  preserving  timed  languages.  Given  a  hybrid  system, 
the  timed  language  is  simply  the  timed  trajectory  of  the  discrete  states.  There¬ 
fore,  the  timed  language  maintains  the  discrete  state  the  system  is  in  as  well  as 
relevant  timing  information. 

This  problem  is  important  for  a  variety  for  reasons.  For  scheduling  multiple 
physical  processes  (such  as  air  traffic  management  systems) ,  the  higher  level  may 
be  simply  interested  in  which  discrete  mode  each  process  is  in  (landing,  holding, 
etc.)  and  when.  Therefore  the  higher  level  (air  traffic  control)  would  like  then  to 
use  the  simplest  possible  model  of  an  aircraft  that  is  compatible  with  the  original 
aircraft  dynamics  but  also  with  the  scheduling  operation.  Furthermore,  the  re¬ 
sults  of  this  paper  can  be  easily  adapted  to  properly  extract  hybrid  abstractions 
from  purely  continuous  systems  [14].  Finally,  the  results  of  the  paper  are  the  fist 
steps  towards  a  more  general  abstraction  methodology  for  hybrid  systems. 

In  order  for  the  abstracted  model  to  generate  the  same  discrete  symbols, 
we  consider  aggregating  only  the  continuous  dynamics.  Abstracting  the  con¬ 
tinuous  dynamics  while  preserving  the  timed  language  requires  the  abstraction 
process  to  be  done  in  manner  that  allows  us  to  detect  all  the  discrete  transi¬ 
tions.  This  places  a  natural  condition  between  the  abstracting  maps,  guards  and 
invariants  of  the  discrete  transitions.  Assuming  that  our  aggregating  maps  sat¬ 
isfy  these  conditions,  we  show  that  hybrid  trajectories  of  the  original  model  can 
be  simulated  by  the  abstracted  model.  Consequently,  the  abstracted  model  also 
generates  the  same  timed  language.  In  general,  the  abstracted  system  is  not  a 
timed  automaton  [2] ,  as  we  may  need  to  preserve  richer  continuous  dynamics  in 
order  to  properly  detect  the  discrete  transitions. 

In  order  to  ensure  that  timed  trajectories  of  the  abstracted  model  are  feasible 
by  the  original  hybrid  model,  we  rely  heavily  on  the  abstraction  results  for  con¬ 
tinuous  systems  [13].  These  results  give  us  constructive  methods  for  extracting 
hierarchies  of  nonlinear  control  systems  while  preserving  exact  time  controllabil¬ 
ity.  Exact  time  controllability  allows  us  to  preserve  a  form  of  timed  reachability. 
Using  these  results,  we  can  place  additional  conditions  on  our  abstracting  maps 
in  order  to  ensure  that  in  each  discrete  location,  the  ability  to  reach  a  certain 
guard  at  the  same  time  can  be  done  at  both  levels  of  abstraction.  This  allows  us 
to  show  that  the  timed  language  generated  at  the  high  level  can  be  implemented 
at  the  lower  level. 

This  paper  is  organized  as  follows  :  In  Section  2,  we  review  the  continuous 
abstraction  methodology  as  presented  in  [11,13].  In  Section  3,  we  define  hybrid 
systems,  and  determine  conditions  under  which  the  hybrid  abstraction  and  the 
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original  hybrid  system  model  can  generate  the  same  timed  language.  Our  con¬ 
structions  are  briefly  illustrated  by  a  simple  example  in  Section  4,  but  the  reader 
is  referred  to  a  more  detailed  application  in  [14].  Section  5  contains  interesting 
issues  for  further  research. 


2  Abstractions  of  Continuous  Systems 

Contrary  to  differential  equations  whose  abstractions  are  characterized  by  very 
strict  conditions,  abstractions  of  control  systems  involve  only  moderate  con¬ 
ditions  due  to  the  nondeterministic  nature  of  control  systems.  In  subsequent 
discussion,  we  assume  the  reader  is  familiar  with  diflPerential  geometric  concepts 
at  the  level  presented  in  [1]. 

2.1  Abstractions  of  Control  Systems 

We  begin  with  an  abstract  definition  of  a  control  system: 

Definition  1  (Control  System).  A  control  system  S  =  {U,F)  consists  of  a 
fiber  bundle  ir  :  U  — >  M  called  the  control  bundle  and  a  smooth  map  F  :  U 
— >  TM  which  is  fiber  preserving,  that  is  tt'  o  F  =  tt  where  n'  :  TM  — >  M  is 
the  tangent  bundle  projection.  Given  a  control  system  S  —  {U,F),  the  control 
distribution  T>  of  control  system  S,  is  naturally  defined  pointwise  by  T>{x)  = 
F{7t~^{x))  for  all  x  G  M. 

The  control  space  U  is  modeled  as  a  fiber  bundle  since  in  general  the  con¬ 
trol  inputs  available  may  depend  on  the  current  state  of  the  system.  On  a  local 
coordinate  chart.  Definition  1  can  be  read  as  -^x  =  f{x,u)  with  u  G  7r“^(a;), 
therefore  recovering  the  traditional  form  of  the  control  system.  Before  intro¬ 
ducing  the  notion  of  abstraction  for  continuous  control  systems,  the  concept  of 
trajectories  of  control  systems  is  required: 

Definition  2  (Trajectories  of  Control  Systems).  A  curve  c  :  I  M, 
I  C  is  called  a  trajectory  of  control  system  S  —  {U,F)  if  there  exists  a  curve 
(F  :  I  — vU  satisfying: 


t:  o(F  =  c 

|c(i)=c,(|)  =  c.(l)  =  F(c^) 

Again  in  local  coordinates,  the  above  definition  simply  says  that  x{t)  is  a 
solution  to  a  control  system  if  there  exists  an  input  u{t)  G  U{x{t))  =  7r~^(x{t)) 
satisfying  ■^x{t)  =  f{x{t),u{t)).  Our  goal  is  to  construct  a  map  :  M  ^  N, 
the  abstraction  map  or  aggregation  map,  that  will  induce  a  new  control  sys¬ 
tem  on  the  lower  dimensional  manifold  N  having  as  trajectories 

where  c  are  S  trajectories.  The  concept  of  abstraction  map  for  continuous  control 
systems  is  defined  as  follows: 
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Definition  3  (Abstraction  Map).  Let  Sm  =  and  Sj^  =  {Un^F^) 

be  two  control  systems  on  manifolds  M  and  N,  respectively.  A  map  (f)  \  M  N 
is  called  an  abstraction  or  aggregation  map  iff  for  every  trajectory  of  Sm, 
(/>(c^)  is  a  trajectory  of  S^.  Control  system  is  called  a  (f)- abstraction  of  Sm- 

The  above  definition  is  clearly  inspired  from  the  notions  of  language  equiva¬ 
lence  and  simulation  of  transition  systems  [10].  From  Definition  3,  it  is  clear  that 
an  abstraction  captures  all  the  trajectories  of  the  original  system,  but  may  also 
contain  redundant  trajectories.  These  redundant  trajectories  are  not  feasible  by 
the  original  system  and  are  therefore  undesired. 

Since  Definition  3  defines  abstractions  at  the  level  of  trajectories,  it  is  difficult 
to  determine  whether  a  control  system  is  an  abstraction  of  another  one,  since 
this  would  require  integration  of  the  control  systems.  One  is  then  interested  in 
a  characterization  of  abstractions  which  is  equivalent  to  Definition  3  but  easily 
checkable.  To  pursue  this,  one  needs  to  introduce  the  notion  of  (^-related  control 
systems. 

Definition  4  ((/>-related  control  systems).  Let  Sm  =  {Um^Fm)  and  Sm  = 
(Bn,  Fn)  be  two  control  systems  defined  on  manifolds  M  and  N,  respectively.  Let 
(j)  :  M  — >  N  be  a  smooth  map.  Then  control  systems  Sm  and  Sm  are  (j)-related 
iff  for  every  x  E  M 


<!>*  {Fu{n^{x))^  C  (1) 

The  notion  of  (^-related  control  systems  is  a  generalization  of  (/►-related  vector 
fields  commonly  found  in  differential  geometry  as  explained  in  [11].  It  is  evident 
that  given  two  systems  that  are  (/>-related  to  a  control  system  their  intersec¬ 
tion  is  also  ^-related.  This  immediately  suggests  that  given  a  control  system 
and  a  map  (j),  there  is  a  minimal  (p-related  control  system,  in  which  case  the 
inclusion  (1)  can  be  replaced  by  equality^.  We  can  now  provide  the  connection 
between  abstractions  and  (/>-related  control  systems: 

Theorem  1  ([12,11]).  Let  Sm  and  Sm  be  control  systems  on  manifolds  M  and 
N j  respectively,  and  cf) :  M  — >  N  a  smooth  map.  Then  Sm  and  Sm  are  (j)-related 
if  and  only  if  Sm  is  a  (j)- abstraction  of  Sm- 

The  control  system  Sm  is  called  the  minimal  (/►-abstraction  of  a  control  system 
Sm  iff  Sm  is  the  minimal  system  that  is  (/)-related  to  Sm- 

For  analytic  control  systems  there  is  a  constructive  method  which  given  a 
control  system  Sm  and  a  map  (p  \  M  N ,  generates  a  (/>- abstraction  Sm-  This 
construction,  which  generalizes  the  construction  for  linear  systems  described 
in  [11],  is  now  briefly  reviewed.  The  reader  is  referred  to  [13]  for  more  details. 

Given  two  distributions  A  and  B  on  manifold  M,  define  a  distribution  [A,  B] 
by  declaring  [A,  B]  (p)  to  be  the  subspace  of  TpM  generated  by  vectors  of  the  form 
[X,  Y](p),  where  X,Y  are  any  two  analytic  vector  fields  in  A  and  B  respectively. 


^  Note  that  this  minimal  element  is  unique  up  to  a  change  of  coordinates. 
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and  [X,  Y]  is  their  Lie  bracket.  By  resorting  to  this  constructive  method,  define 
the  distribution  T>m  as: 


Dm  =  ^  U  Dm  U  [/C,  Dm]  U  [/C,  [X,  Dm]]  U  . . .  (2) 

where  K  is  the  integrable  distribution  Ker{(j)^),  (j)^  is  the  push  forward  map  oUj), 
and  Dm  the  distribution  associated  with  control  system  Sm^  Distribution  Dm 
allows  us  to  construct  the  minimal  (/>-abstraction  on  X  as: 

VN{y)  =  4>,(vM{x))  (3) 

for  any  x  €  If  is  extracted  from  Sm  using  this  canonical  construction, 

then  control  system  will  be  referred  to  as  canonically  (j)-related  to  5m- 

2.2  Controllability  Equivalence 

In  general,  since  the  abstracted  system  is  less  constrained,  the  abstracted  model 
may  allow  evolutions  that  might  not  be  implementable  on  the  original  system. 
However  the  original  system  and  its  abstraction  can  still  be  rendered  equivalent 
regarding  some  properties  of  interest.  In  this  paper,  we  will  focus  on  exact  time 
controllability  which  is  defined  using  the  reachable  sets  of  control  system  Sm' 

Definition  5  (Reachable  set  [7]).  For  each  T  >  0,  and  each  x  in  M,  the  set 
of  points  reachable  from  x  at  time  T,  denoted  by  Reach{XyT),  is  equal  to  the  set 
of  terminal  points  c^(T)  of  Sm  trajectories  that  originate  at  x. 


Definition  6  (Exact  Time  Controllability).  A  control  system  is  said  to  be 
exact  time  controllable  if  for  any  T  >  0,  Reach{x,T)  =  M  for  any  x  E  M. 

Consider  two  systems  Sm  and  Sn  and  a  surjective  map  (j)  \  M  N.  Control 
systems  Sm  and  Sm  are  equivalent  from  an  exact  time  controllability  point  of 
view  if  the  following  property  holds:  there  exists  an  Sm  trajectory  connecting 
xi  £  M  to  X2  £  M  in  time  T  if  and  only  if  there  exists  diS^  trajectory  connecting 
<l){xi)  e  N  to  (j){x2)  £  N  also  in  time  T.  This  property  is  clearly  reminiscent  of 
timed-bisimulations  [10]. 

If  we  assume  that  the  control  system  is  affine  in  the  control,  that  is,  on  local 
charts  it  can  be  written  as: 


k 

F{x,u)  =  f{x)  +  'Y^gi{x)ui  (4) 

i=\ 

then  we  can  characterize  exact  time  controllability  through  the  Lie  algebra  gen¬ 
erated  by  {gi{x),g2{x),  ...,gk{x)}  and  denoted  by  Lieg{SM)- 

Theorem  2  ([7]).  An  analytic  control  system  Sm  control,  as  defined 

in  (4),  is  exact  time  controllable  if  Lieg{SM{x))  =  TxM  for  every  x  £  M. 
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We  defer  the  reader  to  [6,7]  for  further  details  regarding  the  various  notions  and 
concepts  of  controllability.  The  main  theorem  regarding  controllability  equiva¬ 
lence  of  abstractions  (see  [13])  can  now  be  restated  as  follows: 

Theorem  3  (Exact  Time  Controllability  Equivalence).  Let  Sm  and  Sn 

be  two  analytic  control  systems  on  analytic  manifolds  M  and  N,  respectively, 
and  let  N  be  an  embedded  submanifold  of  M.  Let  (j)  :  M  N  be  an  analytic 
surjective  submersion.  If  Sn  is  canonically  (j)-related  to  Sm  and 

Ker{<l)^)  C  Lieg{SM)  (5) 

then  Sn  is  exact  time  controllable  iff  Sm  i'S. 

Equations  (2,3)  and  Theorem  3  provide  a  constructive  way  of  building  con¬ 
tinuous  abstractions  that  propagate  reachable  sets,  and  in  particular  exact  time 
controllability.  When  additional  properties  must  be  propagated,  additional  con¬ 
straints  must  be  imposed  on  the  abstracting  maps. 


3  Hybrid  Control  Abstractions 

Although  hybrid  abstractions  follow  the  same  conceptual  ideas  of  discrete  and 
continuous  abstractions,  their  study  is  somewhat  more  involved  due  to  the  com¬ 
plicated  nature  of  hybrid  trajectories.  We  start  with  a  hybrid  system  model  that 
allows  different  continuous  spaces  in  each  discrete  location. 

Definition  7  (Hybrid  Control  System),  A  hybrid  control  system  is  a  tuple 
H  ~  {X,Xq,  S,Inv,R)  with  the  following  components: 

—  X  is  the  state  space  of  the  hybrid  control  system  and  is  given  by  a  family  of 
smooth  manifolds  X  =  {Mq}g^Q  indexeS  by  a  finite  set  Q.  Each  state  thus 
has  the  form  {x,q),  where  x  €  Mg  is  the  continuous  part  of  the  state,  and 
q  e  Q  is  the  discrete  part. 

-  Xq  =  {Mg  IggQQ  Q  X  is  the  set  of  initial  states. 

~  S:  Q  ^  {{Ug,Fg)  :  [Ug,Fg)  is  a  control  system  on  Mg)  assigns  to  each 
discrete  state  q  £  Q  a  control  system  {Ug,Fg)  which  governs  the  evolution  of 
the  continuous  part  of  the  state.  Thus  in  discrete  location  q,  the  continuous 
part  of  the  state  satisfies  j^x  =  f{x,q,u)  with  u  G  7r“^(n:,^). 

-  Inv:  Q  2^  assigns  to  each  location  q  £  Q  an  invariant  set  Inv(q)  C  Mg. 

—  R  Q  X  X  X  is  a  relation  capturing  the  discrete  jumps. 

Hybrid  systems  are  typically  represented  as  finite  graphs  with  vertices  Q, 
and  edges  E  defined  by 

^  €  Q  X  Q  I  {{x,q),{x' ,q'))  £  R  ior  x  £  Inv{q)  and  x'  £  Inv{q')}. 

^  When  all  the  manifolds  Mq  are  equal,  then  the  state  space  X  is  X  =  M  x  Q. 
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With  each  edge  e  =  {q,q')  ^  E  we  associate  a  guard  set  defined  as 

Guard{e)  =  {x  e  Inv{q)  \  ((x,g),  (x\q'))  G  R  for  some  x'  e  Inv(q')} 

and  a  set- valued  reset  map 

Reset{e,x)  =  {x'  e  Inv{q')  \  ((x,g),  (x',q'))  G  R}. 

Trajectories  of  the  hybrid  system  H  originate  at  any  initial  state  {x,  q)  G  Xq  and 
consist  of  concatenations  of  continuous  flows  and  discrete  jumps.  Continuous 
flows  keep  the  discrete  part  of  the  state  constant  at  g,  and  the  continuous  part 
evolves  over  time  according  to  the  control  system  -^x  =  f{x,q,u),  as  long  as  x 
remains  inside  the  invariant  set  Inv{q).  If  during  the  continuous  flow,  it  happens 
that  X  G  Guard{e)  for  some  e  =  (g,  q')  G  E,  then  the  edge  e  becomes  enabled. 
The  state  of  the  hybrid  system  may  then  instantaneously  jump  from  (x,  q)  to 
any  (x',q')  with  x'  G  Reset{e,x).  Then  the  process  repeats,  and  the  continuous 
part  of  the  state  evolves  according  to  the  control  system  =  f{x,q',u).  We 
shall  therefore  assume  that  a  trajectory  of  an  hybrid  control  system  is  a  map^  ^ 
from  a  time  set  T  to  the  state  space  X  =  of  H,  that  is: 


$  •  T  — >  {Mq}q^Q 

T  t->  (x(r),g(r))  (6) 

An  abstracting  map  for  hybrid  systems  can  now  be  defined  in  the  same  way  it 
was  defined  for  continuous  systems. 

Definitions  (Abstraction  Map).  Let  Hx  =  {X,Xo,Sx,InvxzRx)  o,nd 
Hy  —  {Y,Yo,  Sy ,  Invy ,  Ry)  be  two  hybrid  control  systems  with  X  =  {Mq}q^Q 
and  Y  =  {Npjp^p.  A  map  (j)  :  X  Y  is  called  an  abstraction  or  aggregation 
map  iff  for  every  trajectory  of  Hx,  is  a  trajectory  of  Hy. 

Even  though,  we  are  interested  in  general  abstracting  maps,  we  now  focus  on 
a  subclass  of  abstracting  maps  that  are  suitable  for  preserving  timed  languages. 

3.1  Timed  Language  Generated  by  a  Hybrid  System 

In  this  paper  we  shall  focus  on  abstractions  that  render  the  original  system 
and  its  abstraction  equivalent  regarding  the  timed  language  they  can  generate. 
The  timed  string  corresponding  to  a  trajectory  ^(r)  =  (a:(r),g(r))  of  an  hybrid 
control  system  is  simply  given  by  q{t).  Naturally  q{t)  can  be  regarded  as  a  timed 
string"^  since  it  can  be  written  in  the  more  usual  form  {(t,  The  timed 

language  generated  by  an  hybrid  control  system  is  therefore  defined  as: 

^  When  multiple  discrete  jumps  in  zero  time  are  allowed,  a  more  complex  notion  of 
time  is  required  to  regard  an  hybrid  trajectory  as  a  map,  see  for  example  [9]. 

The  string  s  =  q{t)  can  be  transformed  to  retain  only  the  discrete  states,  and  the 
first  instance  of  time  at  which  the  system  has  changed  discrete  state.  The  results 
presented  in  this  paper  are  however  independent  of  that  transformation. 
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Definition  9  (Timed  language  of  a  hybrid  system).  Let  H  be  a  hybrid 
control  system.  The  timed  language  generated  by  H  and  denoted  by  Eh  is  given 
by  all  the  strings  q{t),  where  q(t)  is  the  discrete  part  of  an  hybrid  trajectory 
{(r)  =  (x(r),g(r))  ofH. 

With  this  notion  of  timed  language,  timed  language  equivalence  between  two 
hybrid  system  requires  the  discrete  behavior  of  the  hybrid  abstraction  to  be  equal 
to  the  discrete  behavior  of  the  original  system.  Therefore  aggregation  can  only 
happen  on  the  continuous  part  of  the  hybrid  system.  We  will  therefore  restrict 
the  class  of  abstracting  maps  to  the  following  form: 

^  •  {^q}qeQ  — >  {^q}qeQ 

(f>{x,q)  =  {(l)(x),q)  (7) 

that  is,  if  <j>  is  written  as  (/>  =  then  (I)q  is  the  identity  map  on  Q  =  P. 

Even  though  for  continuous  systems  we  can  always  extract  abstractions  that 
preserve  trajectories,  for  hybrid  control  systems  additional  constraints  must  be 
imposed  on  the  abstracting  map  to  ensure  timed  language  equivalence.  This  is 
because  the  discrete  dynamics  rely  heavily  on  certain  sets,  such  as  the  guards 
and  the  invariants,  and  we  have  to  ensure  that  these  sets  are  abstracted  correctly 
at  the  higher  level. 

3.2  Propagating  Guards  and  Invziriants 

Let  us  zoom  into  a  discrete  state  and  consider  the  relevant  sets  which  trigger 
the  discrete  dynamics,  namely  the  guards  and  the  invariants.  Timed  language 
equivalence  requires  that  these  sets  must  be  aggregated  in  a  consistent  way. 

Figure  1  represents  the  state  space  of  the  original  system  with  the  guard 
defined  by  a  relation  of  the  type  X2  >  const.  When  performing  an  abstraction 
using  the  map  0(xi,X2)  =  :r2,  in  the  abstracted  system  it  is  still  possible  to 
determine  if  the  continuous  part  of  the  trajectory  belongs  or  not  on  the  guard. 
No  information  required  by  the  discrete  dynamics  was  lost  in  the  abstracting 
process.  However  if  the  abstracting  map  is  j){xi,X2)  ==  Xi  it  is  no  longer  possible 
to  determine  if  the  continuous  part  of  the  trajectory  belongs  or  not  to  the  guard, 
therefore  it  is  not  possible  to  generate  the  same  timed  language. 

The  essential  property  to  be  propagated  is  therefore  the  ability  to  distinguish 
between  sets  (p{A)  and  ^(P)  in  the  abstracted  system  if  and  only  if  it  is  possible 
to  distinguish  between  relevant  sets  A  and  B  in  the  original  system.  The  relevant 
sets  can  be  encoded  in  a  partition  of  the  state  space,  where  each  equivalence  class 
of  the  partition  corresponds  to  a  possible  combination  of  guards  and  invariants. 
The  required  partition  can  be  modeled  as  a  map  defined  as: 

-^D  (8) 

where  D  is  a  finite  set.  We  assume  that  the  map  results  in  a  topologically 
well  behaved  partition^.  Partition  propagation  can  now  be  defined  as: 

^  For  example,  the  partition  can  be  a  subanalytic  stratification  [8]. 
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Xi 


Fig.  1.  Detecting  a  guard. 


Definition  10  (Partition  Propagation).  An  abstracting  map  cj)  :  M  N 
propagates  a  partition  there  exists  a  partition  on  N  defined  by  a  map 

:  N  D  such  that  the  following  diagram  commutes. 


(9) 


or  equivalently  iff  ^m{^)  =  ^ 

Note  that  propagating  the  partitions  is  stronger  than  preserving  the  partition 
which  only  requires  that  =  ^m(^2)  =>  ^  ^  <^(^2)  and 

allows,  for  example,  merging  two  equivalence  classes  into  a  single  equivalence 
class  in  This  is  not  a  desirable  situation  since  the  ability  to  distinguish 
between  the  two  equivalence  classes  is  lost. 

Although  Definition  10  captures  the  fundamental  property  that  the  abstract¬ 
ing  map  should  possess  it  does  not  characterize  it  directly.  A  characterization  is 
given  in  the  following  proposition: 

Proposition  1.  An  abstracting  map  (j)  :  M  — >  N  propagates  a  partition 
iff  the  preimage  under  (j)  of  a  point  y  e  N  is  totally  contained  in  a  single 
equivalence  class,  equivalently,  if  for  all  y  £  N  there  exists  one  and  only  one 
d  £  D  such  that  ^ 

Proof  (Sufficiency)  We  proceed  by  contradiction.  Suppose  that  ^m{^)  =  ^ 

(t){x)  and  there  exist  two  different  elements  a,b  e  M  that  belong  to  two  different 
equivalence  classes,  that  is  ^  Admit  further  that  they  are 

mapped  into  the  same  point  in  N,  0(a)  =  0(6).  We  have  that 
but  since  0(a)  —  0(6),  !P’jvo0(a)  =  ^j\fO<f{b)  =  Therefore  =  tf'M(6), 

a  contradiction. 
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(Necessity)  We  define  explicitly  the  map  as  =  ^m{^)  for  all  x  E  (t>~^{y) 

which  is  well  defined  since  (t>~^{y)  is  contained  in  a  single  equivalence  class. 

□ 

Proposition  1  states  partition  propagation  conditions  explicity  on  the  ab¬ 
stracting  map  (j),  but  they  are  very  difficult  to  check  in  general.  However  it  is 
rather  intuitive  that  a  sufficient  condition  for  partition  propagation  is  symmetry, 
as  expressed  in  the  next  proposition. 

Proposition  2.  Suppose  that  the  partition  on  manifold  M  is  invariant  un¬ 
der  the  action  of  a  group  G,  then  the  abstracting  map  cj)  defined  as  the  projection 
from  the  manifold  M  to  the  orbit  space  M/G  propagates  the  partition 

Proof  If  the  equivalence  classes  are  invariant  under  G  action,  then  the  orbit 
through  the  point  xq^  namely  Oxq  =  {x  G  M  :  x  =  qxq  is  contained  in 

a  equivalence  class.  Since  the  preimages  under  (j)  are  precisely  the  sets 
the  conditions  of  Proposition  1  are  satisfied.  □ 

In  fact,  symmetry  is  also  a  necessary  condition  when  more  structure  is  im¬ 
posed  on  the  set  M  and  the  map  <j).  To  study  general  nonlinear  abstracting  maps 
we  consider  that  M  and  N  are  smooth  manifolds  and  that  the  abstracting  map 
j)  is  a  smooth  surjective  submersion.  Resorting  to  this  differentiable  structure, 
Proposition  1  specializes  to: 

Proposition  3.  A  smooth  surjective  submersion  (j)  :  M  N  between  smooth 
manifolds  propagates  a  partition  if  and  only  if  the  partition  equivalence 
classes  are  invariant  under  Ker{(p^). 

Proof  (Sufficiency)  The  vectors  in  Ker{<f^)  span  an  involuntive  distribution 
which  has  constant  rank  at  every  x  e  M  since  the  map  is  a  submersion.  By 
Frobenius  theorem  [1]  there  exists  an  integrating  manifold  that  can  be  described 
as  the  action  of  with  p  =  on  M  given  by  7  =  o  ^2(^2)  o  • . .  o 

(l>p{tp).  Each  (pi{ti)  is  the  flow  of  the  vector  field  from  the  generators  of  /C,  that 
is  /C  =  Span{Z^ ,  Z'^ , . . . ,  Z^}.  The  partition  equivalence  classes  are  therefore 
invariant  under  this  action  and  by  Proposition  2  the  partition  is  propagated. 

(Necessity)  The  preimage  of  a  point  1/  E  by  is  a  smooth  submanifold  of  M 
when  the  derivative  of  </>,  is  surjective,  which  is  the  case  since  0  is  an  submersion. 
The  tangent  space  of  the  submanifold  is  given  by  the  vectors  X  E  TM 

that  belong  to  Ker{(j)^).  Since  the  partition  is  propagated  the  preimage  of  a 
point  y  e  N  hy  (f)  is  totally  contained  inside  a  partition  equivalence  class  and 
therefore  the  partition  equivalence  classes  are  invariant  under  Ker{(j)f).  □ 

The  above  characterizations  of  the  abstracting  maps  are  critical  in  order  to 
propagate  discrete  trajectories  from  the  original  hybrid  control  system  to  the 
abstracted  one  while  ensuring  timed  language  equivalence. 
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3.3  Hybrid  Abstractions 

Given  a  hybrid  system,  Hx  and  an  abstracting  map  cj),  we  now  present  a  con¬ 
struction  that  generates  an  hybrid  abstraction  Hy.  The  abstraction  process  de¬ 
pends  on  the  observation  that  the  continuous  dynamics  in  a  particular  discrete 
state  is  essentially  decoupled  from  the  continuous  dynamics  in  the  other  discrete 
state,  the  only  link  being  given  by  the  Reset  map.  It  is  therefore  possible  to  use 
a  different  abstracting  map  (j)q  in  each  discrete  state  g  G  Q  of  the  hybrid  system 
Hx-  More  formally: 

Definition  11  (Construction  of  hybrid  abstractions).  Consider  hybrid 
control  system  Hx  —  {X^XQ^Sx^Invx.Rx)  X  =  {Mq\q^Q  and  consider 
the  collection  of  maps  ^  —  {(t>q}qeQy  4>q  '  ^q  — ^  Xq.  The  resulting  hybrid  ab¬ 
straction  Hy  —  {Y,Yo,  Sy  ^  Invy  ^  Ry)  is  a  tuple  consisting  of: 

—  For  all  q  ^  Q,  Nq  —  (t)q{Mq),  therefore  the  state  space  isY  =  {Nq}q^Q. 

-Yo  =  where  N°  = 

—  Sy  is  a  function  that  maps  each  q  £  Q  to  the  minimal  ^q- abstraction  of  the 
corresponding  control  system  Sx{q)  using  the  canonical  construction  (2,3). 

-  Invy{q)  =  (f)q{Invx{q))- 

-  Ry  =  {{{y,q),{y',q'))  eY  xY  :  {y,q)  =  (t>qix,q)  A  {y\q')  =  (t>q>(x' ,q')  A 
{{x,q),(x',q'))  £  Rx}-  More  specifically  we  have 

—  Guardy{e)  =  (j)q^{Guardx{€)) 

—  Resety(e,Xi)  =  (pq.  o Reset x{e,<l)qf{xi))  for  all  e  =  {qi^qj)  £  E,  x  £  M . 

Therefore  the  discrete  state  space  remains  unaltered  and  only  the  continuous 
state  space  is  aggregated  from  Mq  to  Nq  is  each  discrete  location  q  £  Q,  and 
similarly  for  the  set  of  initial  conditions.  The  continuous  control  system  Sx{q) 
is  replaced  by  its  minimal  </>g-abstraction.  The  new  invariant  on  each  location 
q  £  Q  IS  the  image  of  the  initial  invariant  under  <f)q,  that  is  (l)q{Invx{q))-  The 
reset  relation  Ry  is  the  image  of  the  reset  relation  Rx  by  the  abstracting  map 
resulting  in  the  new  guards  being  the  image  of  the  initial  guards  by  the  abstract¬ 
ing  map.  The  reset  maps  Resety  are  given  by  the  image  under  (j)q^  of  the  reset 
maps  Reset X  evaluated  at  every  point  of  the  set  valued  map  The  main 
result  relating  hybrid  abstraction  constructed  through  Definition  11  and  timed 
language  equivalence  can  now  be  stated  as  follows: 

Theorem  4  (Timed  language  equivalent  hybrid  abstractions).  Let  Hx 

and  Hy  be  hybrid  control  systems  and  suppose  Hy  is  obtained  from  Hx  using 
Definition  11.  If  the  family  of  maps  ^  =  {(f>q}q^Q  is  such  that  the  invariants  and 
guards  in  each  discrete  location  q  £  Q  are  invariant  under  Ker{(j)q^)  then  Hy  is 
a  ^-abstraction  of  Hx- 

If  furthermore  Ker{(j)q^)  C  Lieg{SM{q))  for  each  q  £  Q  then  Hx  ond  Hy 
generate  the  same  timed  language. 

Proof.  To  show  that  Hy  is  a  ^-abstraction  of  Hx  we  need  to  show  that  for  every 
trajectory  —  (x{T),q{r)),  is  a  trajectory  of  Hy.  For  any  trajectory 

{x(T),q{T))  of  Hx,  {x{0),q{0))  £  Xq,  therefore  ^{a:(0),  g(0))  -  (^^(o),  g'(O))  G  Yq 


512  P.  Tabuada  and  G.J.  Pappas 


Fig.  2.  Hybrid  control  system  Hx- 


since  As  long  as  the  trajectory  flows  continuously  on  a 

state  q  ^  Q,  x{r)  is  a  trajectory  of  Sx(q),  therefore  y(r)  is  a  trajectory  of 
Syiq)  since  Sy{p)  is  (^^-related  to  Sx{q)  and  x{t)  €  Invx(q)  implies  y(T)  G 
Invyiq)  by  construction  and  partition  propagation.  When  x{t)  enters  a  guard 
Guardx{e),  y{r)  enters  Guardy{e)  by  construction  and  partition  propagation. 
If  the  hybrid  control  system  Hx  jumps  from  location  qi  to  location  qj  then  Hy 
can  also  take  the  same  transition  since  the  finite  graphs  of  Hy  and  Hx  are  equal 
and  the  corresponding  transitions  become  enabled  at  the  same  time.  After  the 
jump  x{r)  G  Resetx{e,x')  and  therefore  ?/(r)  G  Resety{e^y')  by  construction 
of  Resety.  Since  the  trajectory  is  composed  of  continuous  flows  and  jumps 
and  Hy  simulates  both,  a  finite  induction  argument  on  the  number  of  jumps 
concludes  the  proof. 

To  show  timed  language  equivalence  it  suffices  to  show  that  hybrid  control 
system  Hx  is  capable  of  simulating  the  continuous  part  of  every  Hy  trajectory 
since  both  systems  have  the  same  finite  graph.  This  is  now  a  direct  consequence 
of  using  the  minimal  control  abstraction  Si^{q)  of  control  system  SM{q)  in  each 
discrete  location  g  G  Q  as  Theorem  3  asserts  that  both  control  systems  are  exact 
time  controllability  equivalent.  □ 


4  Example 

We  illustrate  our  results  by  a  simple  example.  Consider  the  hybrid  control  system 
Hx  displayed  in  Figure  2.  Using  as  abstracting  maps  <))q^  =  xiX2  and  (pq^  =  xi 
we  extract  the  timed  language  equivalent  abstraction  presented  in  Figure  3.  Due 
to  space  restrictions,  we  shall  present  the  details  regarding  state  ^2-  We  start  by 
noting  that  Inv{q2)  is  invariant  under  Ker{(j)q^^)  =  K  =  -^  since  K  is  every¬ 
where  tangent  to  the  surfaces  Xi  =  const.  The  guard  is  given  the  complement 
of  the  invariant  and  is,  therefore,  also  invariant  under  K.  The  next  step  is  to 
determine  if  (t>q^  satisfies  Theorem  3  conditions,  but  this  is  automatically  true 
since  K  =  5i(x),  and  therefore  K  G  Lieg{gi{x)}  =  {51(2:)}.  The  new  dynamics 
in  each  location  can  be  determined  through  the  construction  (2,  3).  Writing  the 
dynamics  as  i:  =  f(x)  -h  g{x)u  we  compute  [K,  /]  =  Xi  =  ^  +  2xiX2^  and 
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Fig.  3.  Hybrid  abstraction  Hy  of  the  hybrid  control  system  Hx- 


[K,Xi\  -  X2  ~  2x1-^.  However  X2  in  linearly  dependent  on  g  so  that: 

Pm  =  {/,9,^i}  (10) 

Computing  the  pushforward  by  ^,2  df  T^m{x)  we  get: 

<)>,,.(Pm(x))  ={(x2+a:f)^,^}  (11) 

In  N  coordinates,  (given  by  x'),  Xi  equals  x'  and  X2  is  now  regarded  as  a  control 
input  V.  The  new  dynamics  is  then  given  by  x'  =  l+x'^+t;  and  after  introducing  a 
new  control  input  given  by  u'  =  t;+1+x'^  we  get  finally  x'  —  u' .  The  invariant  on 
N  becames  =  x'  <  0  and  the  guard  reads  x'  >  0.  To  determine  the  new  reset 
map  one  computes  4>~^(x')  =  {(xi,X2)  £  M  :  xi  =  x'Ax2  G  E}.  Using  this  data 
the  reset  map  of  the  hybrid  automaton  Hx  isxj  :=  — 1  —  |x^|  =  — 1— x^  (since  the 
guard  is  only  enable  for  xi  >  0)  and  X2  :=  1  +  [0,  +oo[=  [1,  +oo[.  Aplying  (j)q^  to 
this  reset  maps  gives  the  new  reset  map  x'  :=  (-1— x')([l,  +oo[)  =]  -oo,  ~1— x']. 

Note  how  in  this  case  the  nonlinear  dynamics  could  be  simplified  in  such  a 
way  that  HyTech  [5]  or  other  similar  tool  can  be  used  to  analyze  the  resulting 
abstraction.  For  a  more  complicated  example  which  extracts  a  hybrid  abstraction 
from  a  purely  continuous  system,  the  reader  is  referred  to  [14], 


5  Conclusions 

In  this  paper,  we  have  considered  the  problem  of  extracting  hybrid  abstractions 
from  hybrid  control  systems  while  preserving  timed  languages.  Generalizing  the 
results  of  this  paper  to  more  general  abstracting  maps  and  more  general  prop¬ 
erties  is  clearly  important.  Different  properties  may  require  different  conditions 
on  the  abstracting  maps,  as  well  as  different  compatibility  conditions  between 
the  abstracting  maps  and  the  guards,  invariants,  and  continuous  dynamics. 
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